Understanding Trust and Security Processes in the Open Source Software Ecosystem
Dominik Wermke
dwermke.com/enigma
USENIX Enigma 2023
Dominik Wermke
Security & Privacy Researcher
CISPA Helmholtz Center for Information Security
dwermke.com/enigma

The Supply Chain
is under
Attack!

Targeting Developers


Tech Monitor, 2023-01-03

Developers as the weakest Link

[A]n unauthorized third party leveraged malware deployed to a CircleCI engineer’s laptop in order to steal a valid, 2FA-backed SSO session.
CircleCI

Where does Research get involved?

Usable Security & Privacy
Security & Privacy
Security & Privacy
Security & Privacy
Usable Security & Privacy
If it's not the users fault ...
  • Log4Shell vulnerability
  • Heartbleed, Cloudbleed
  • Shellshock vulnerability, Bashdoor

... is it the developers fault?

Even the bad guys fail

Usable Security & Privacy
Usable Security & Privacy for Developers
How can we help ...
  • Developers
  • Maintainers
  • Administrators
  • ...

Why Experts?

Why Experts?

  • Users
  • Administrators
  • Maintainers
  • Developers
  • Project Leads

Why Experts?

  • Users
  • Administrators
  • Maintainers
  • Developers
  • Project Leads

Why Experts?

  • 1,000,000s of end users
  • 1000s of deployments
  • 100s of lines of code
  • 10s of projects

Experts but not Security Experts


The New Stack

Technology Only Can't Solve The Problem

Technology Only Can't Solve The Problem

The company said intruders had gained access to its cloud database and obtained a copy of the data vaults of tens of millions of customers by using credentials and keys stolen from a LastPass employee.
New York Times, 2023-01-05

Technology Only Can't Solve The Problem

Given 89 scenarios, about 40 per cent of the computer programs made with the help of Copilot had potentially exploitable vulnerabilities.
The Register

Open Source Software

  • Usage: Most projects on their platform rely on some form of Open Source Software
  • Connected: Median of 683 transitive dependencies in the npm ecosystem
Why you should care about Open Source

Open Source Software

Open Source Supply Chain

Open Source code appears as foundation, glue, or during the build process in many software projects.

Committed to Trust: 27 interviews with maintainers of OSS [IEEE S&P 2022]
Reproducible Builds: 24 interviews with reproducible builds stakeholders [Ongoing]
Open Source Components: 25 interviews with company stakeholders using OSS [Ongoing]

Committed to Trust

A Qualitative Study on Security & Trust
in Open Source Software Projects

Dominik Wermke *
CISPA
Noah Wöhler
CISPA
Jan H. Klemmer
Leibniz University Hannover
Marcel Fourné
MPI-SP
Yasemin Acar
George Washington University
Sascha Fahl
CISPA

Research Questions

  • RQ1:
    How are open source projects structured behind-the-scenes?
  • RQ2:
    If and what guidance and policies are provided by open source projects?
  • RQ3:
    How do open source projects approach security and trust challenges?

Interviews

27
Project Leads, Maintainers, Contributors

Demographics

Highest project role of participants
Contributors - Maintainers - Leaders - Owners

Demographics

Highest project role of participants
Contributors (4) - Maintainers (3) - Leaders (7) - Owners (9)
Other: 4

Results

Trust
Security
Processes

Trust

Trust
Security
Processes

Trust Incidents

Trust incidents encountered in the past:
None --- Some / Other

Trust Incidents

Trust incidents encountered in the past:
None (20) --- Some / Other (7)

Trust Incidents

Trust incidents encountered in the past:
  • Drive-by cryptocurrency miner commits
  • Failed background checks
  • Pro-active block after potential SSH key theft

Trust Processes

Summary
  • Most use some form of meritocracy for establishing trust with new contributors.
  • Some even assume trustworthiness by default to facilitate first-time contributions.
  • The majority never experienced a trust incident in their projects and did not establish specific trust incident strategies.

Security

Trust
Security
Processes

Past Incidents

Security incidents encountered in the past:
None --- Some / Other

Past Incidents

Security incidents encountered in the past:
None (16) --- Some / Other (11)

Past Challenges

Most commonly encountered security challenges
(That not necessarily resulted in an incident)
  • (15) Suspicious or low quality commits
  • (8) Vulnerability introduced by dependencies

Security Challenges

Summary
  • Few projects have experienced an outright security incident
  • Many of our participants were familiar with suspicious or low quality commits and potential vulnerabilities introduced by dependencies

Processes

Trust
Security
Processes

Guidance

Most mentioned guidances
  • (14) Guidance for contributing to the project
  • (13) Programming language-specific guidance
  • (8) General guidance for project setup and infrastructure

Guidance

Somebody would have to write the guide, and I am the only one who can write it. I mean, there is nobody paid to write it and I am also not paid to write it.

Processes

Summary
  • Participants diverge on their opinions regarding the helpfulness of (written) guidance
  • Security policies: larger projects mentioned dedicated security teams, smaller projects mentioned a security contact channel
  • Most projects included some type of disclosure policy or at least contact for security issues

Committed to Trust

Recap
  • Projects are highly diverse in deployed security measures, trust processes, and motivations
  • Growing scope and contributors → growing needs for security and trust processes
  • Smaller projects handle security and trust incidents "as they happen"

Reproducible Builds

To what extent should one trust a statement that a program is free of Trojan horses?
Perhaps it is more important to trust the people who wrote the software.
~ Ken Thompson

Reproducible Builds

  • An important step of trusting the people that wrote the software, is to also be able to trust the resulting artifacts
  • Reproducible builds are a solution to this problem

Challenges

  • Timestamps (current)
  • File order and build directories
  • Any build randomness

Interviews

24
Reproducible Builds Experts

Preliminary Findings

  • Motivation: "Same input should always compile to same output" and broken expectations

Preliminary Findings

  • Experiences: "Receptive upstream projects" and "Patience and Communication"

Preliminary Findings

  • Common Obstacles: Date and other times, build directory name inclusion

Open Source Components in Industry

Security Challenges for Industry

  • Other side of the open source supply chain
  • How do Industry projects deal with challenges of including open source components?
  • Investigate their projects' processes, decisions, and considerations

Research Questions

  • RQ1:
    How are Open Source Components included in companies’ tech stacks in terms of position, importance, and security effects?

Research Questions

  • RQ2:
    What are companies’ awareness, experiences, and attitudes regarding the security of including external open source code?

Research Questions

  • RQ3:
    If and how do stakeholders make decisions and considerations around security and trust challenges of including Open Source Components?

Interviews

25
Industry Stakeholders

Preliminary Findings

Security Challenges

Projects that encountered security challenge due to OSC
No --- Some / Other

Security Challenges

Projects that encountered security challenge due to OSC
No (1) --- Some / Other (24)

Selection Criteria

Most common metrics for selecting OS projects
  • (16) Popularity measure like downloads or GitHub stars
  • (11) Large and active community
  • (10) Activity measures like commit frequency and recent releases
  • (10) Specific features

Internal Mirrors

Projets that use (at least partially) internal mirrors:
Yes --- No / Other

Internal Mirrors

Projets that use (at least partially) internal mirrors:
Yes (14) --- No / Other (11)

"Not Your Average Supply Chain"

  • Some participants mentioned that their management doesn't seem to fully grasp the benefits and challenges of open source software
  • While other support utilized open source components by donating, e.g., money, developer-hours, knowledge, or code
  • Few mentioned even following the guideline to always contribute back

Takeaways

Securing a Bowl of Spaghetti

  • Software supply chain analogy: (wrongly) conveys linear relations
  • With clear start (producer) and end (consumer) endpoints
  • In reality: a giant bowl of spaghetti
  • Some companies focus on stuff on their plate
Committed to Trust: 27 interviews with maintainers of OSS [IEEE S&P 2022]
Reproducible Builds: 24 interviews with reproducible builds stakeholders [Ongoing]
Open Source Components: 25 interviews with company stakeholders using OSS [Ongoing]

Takeaways

  • Open Source:
    Not your average supply chain
  • Developers:
    Weakest link in the software supply chain
  • Solution:
    Involve the developers!

Understanding Trust and Security Processes in the Open Source Software Ecosystem


dwermke.com/enigma
Committed to Trust: A Qualitative Study on Security & Trust in Open Source Software Projects
Wermke et al., IEEE S&P 2022.
Reproducible Builds
[Ongoing]
Open Source Components in Industry Projects
[Ongoing]
Understanding Trust and Security Processes in the Open Source Software Ecosystem
Dominik Wermke
dwermke.com/enigma
USENIX Enigma 2023