Committed to Trust

Committed to Trust

43rd IEEE Symposium on Security and Privacy, 2022

Committed to Trust

A Qualitative Study on Security & Trust
in Open Source Software Projects

Dominik Wermke *
CISPA
Noah Wöhler
CISPA
Jan H. Klemmer
Leibniz University Hannover
Marcel Fourné
MPI-SP
Yasemin Acar
George Washington University
Sascha Fahl
CISPA
Dominik Wermke
Security & Privacy Researcher
CISPA Helmholtz Center for Information Security
Why you should care about Open Source

Open Source Software

Open Source Supply Chain

Open Source code appears as foundation, glue, or during the build process in many software projects.

State of the Octoverse 2020
  • Usage: Most projects on their platform rely on some form of OSS
  • Connected: Median of 683 transitive dependencies in the npm ecosystem
Software Supply Chain
Malicious Attacker
Broken Software Supply Chain

Protestware

To what extent should one trust a statement that a program is free of Trojan horses. Perhaps it is more important to trust the people who wrote the software.
– Ken Thompson in “Reflections on Trusting Trust”

Research Questions

  • RQ1:
    How are open source projects structured behind-the-scenes?
  • RQ2:
    If and what guidance and policies are provided by open source projects?
  • RQ3:
    How do open source projects approach security and trust challenges?
So how could you research Open Source
So how could you research Open Source
Just ask them :)

Recruitment
Channels

  • Stratified Github Sample
  • Personal Network
  • Recommendations
  • ...

Cold Emailing

  • Public Channels
  • General Spam
  • Asking for their time

Email


				Dear `[name]`,
				
				Best regards,
				`[researcher]`
					
Personalized Greeting (low number of messages)

				Dear `[name]`,
				
				We are a group of researchers and contacting you due to your involvement
				in the `[project name]` project on GitHub.
				We are interested in how popular and active open source project
				communities tackle trust & security and would love to interview
				you about it.
				
				Best regards,
				`[researcher]`
					
Summary as intro

				Dear `[name]`,
				
				We are a group of researchers and contacting you due to your involvement
				in the [project name] project on GitHub.
				We are interested in how popular and active open source project
				communities tackle trust & security and would love to interview
				you about it.
				
				We are not trying to sell anything, you & the project would be
				treated completely anonymously.
				We hope your answers and our findings will help with further
				improving security & trust procedures in the open-source community.
				
				Best regards,
				`[researcher]`
					
Pitch, Concerns, and Benefits

				Dear `[name]`,
				
				We are a group of researchers and contacting you due to your involvement
				in the [project name] project on GitHub.
				We are interested in how popular and active open source project
				communities tackle trust & security and would love to interview
				you about it.
				
				We are not trying to sell anything, you & the project would be
				treated completely anonymously.
				We hope your answers and our findings will help with further
				improving security & trust procedures in the open-source community.
				
				If an interview sounds interesting to you, feel free to check out
				our landing page for this research with more information:
				`[landing page URL]`
				
				Best regards,
				`[researcher]`
						
More info (great for sharing)

				Dear `[name]`,
				
				We are a group of researchers and contacting you due to your involvement
				in the [project name] project on GitHub.
				We are interested in how popular and active open source project
				communities tackle trust & security and would love to interview
				you about it.
				
				We are not trying to sell anything, you & the project would be
				treated completely anonymously.
				We hope your answers and our findings will help with further
				improving security & trust procedures in the open-source community.
				
				If an interview sounds interesting to you, feel free to check out
				our landing page for this research with more information:
				`[landing page URL]`
				
				Simply respond to this email with your preferred time slots and
				we will work something out.
				
				Lastly, we are very sorry to take up your valuable time with this email.
				Regrettably, cold emailing is an approach to reach a more diverse set
				of projects. If you do not want to participate, please accept our
				deepest apologies and simply ignore this email,
				we will not contact you again.
				
				Best regards,
				`[researcher]`
					
Apology, Explanation, and Opt-out

Community Channels


				Hi `[project name]` community,
				
				We are a group of researchers from `[affiliations]` and
				we are interested in how open source projects tackle
				trust & security and would love to interview someone
				from the community about it.
				
				Our Landing page: `[landing page URL]`
				
				You can message me directly here on `[platform]` if you
				have any questions or want to schedule an interview!
				
				Thanks for your time,
				`[researcher]`
					
  • Moderator

    (12:14PM, Today)

    Hmm, nobody ever contacted us for posting something like this before 🤔
  • Sure, go ahead.
    But we decided to update the rules for the future 🔨🔨
Sorry for burning this research channel :(

Interviews

27
Participants

Demographics

Highest project role of participants
Contributors - Maintainers - Leaders - Owners

Demographics

Highest project role of participants
Contributors (4) - Maintainers (3) - Leaders (7) - Owners (9)
Other: 4

Distribution

Contributors are often highly distributed

But to be honest, I don’t really mind. As long as one has the same interests, it’s still easy to collaborate if you have the same goal.

Interviews

Project Demographics
Security Challenges
Guidance & Policies
Project Structure
Release & Updates
Roles & Responsibilities
Trust Processes
Opinions & Improvements

Interviews

Project Demographics
Security Challenges
Guidance & Policies
Project Structure
Release & Updates
Roles & Responsibilities
Trust Processes
Opinions & Improvements

Security Challenges

Security Challenges
Guidance & Policies
Trust Processes
Improvements

Past Incidents

Security incidents encountered in the past:
None --- Some / Other

Past Incidents

Security incidents encountered in the past:
None (16) --- Some / Other (11)

Past Challenges

Most commonly encountered security challenges
(That not necessarily resulted in an incident)
  • (15) Suspicious or low quality commits
  • (8) Vulnerability introduced by dependencies
[T]here’s definitely been people that have intentionally tried to put malicious code in projects, but it’s always very easy to spot immediately. It’s like those spam emails where they have bad grammar and stuff

Hypocrite Commit

Opinion of the "hypocrite commit" incident generally:
positive --- mixed --- negative

Hypocrite Commit

Opinion of the "hypocrite commit" incident generally:
positive (0) --- mixed (7) --- negative (16)
No opinion: 4

Hypocrite Commit

Opinion of the "hypocrite commit" incident generally:
I do understand both sides of this […] It would be much better if this kind of research was done in cooperation with somebody at the Linux kernel, who knew that it’s happening and without disclosing that to a lot of people.

Guidance & Policies

Security Challenges
Guidance & Policies
Trust Processes
Improvements

Guidance

Most mentioned guidances
  • (14) Guidance for contributing to the project
  • (13) Programming language-specific guidance
  • (8) General guidance for project setup and infrastructure

Guidance

Somebody would have to write the guide, and I am the only one who can write it. I mean, there is nobody paid to write it and I am also not paid to write it.

Trust Processes

Security Challenges
Guidance & Policies
Trust Processes
Improvements

Trust Incidents

Trust incidents encountered in the past:
None --- Some / Other

Trust Incidents

Trust incidents encountered in the past:
None (20) --- Some / Other (7)

Trust Incidents

Trust incidents encountered in the past:
  • Drive-by cryptocurrency miner commits
  • Failed background checks
  • Pro-active block after potential SSH key theft

Opinions & Improvements

Security Challenges
Guidance & Policies
Trust Processes
Improvements

Reputation

Internal: With one exception, all participants reported a high internal reputation of their projects
Amongst the people on the project, everybody trusts it a lot.
We follow very, very high standards there, mainly because we have a few people who are very, very keen on that.

Reputation

External: The same generally holds true for the external reputation, although many participants are unsure about the actual awareness of the project outside of their community. Overall, our participants appear to take pride in their projects, but are quite humble about their importance and reach in the OSS ecosystem.

Improvements

Suggested improvements mainly require
  • (15) More person-hours
  • (9) More money
  • (9) Different infrastructure

Improvements

Technical Debt
If I could, I would write the entire stack myself.
[…] I would rewrite a lot of the code. That’s just a historical thing, because it has already become big and complex […] It’s just like building a house; you’d have to build it three times before it becomes good.
“What I’d like to do is oxidize [the project] over time, to integrate Rust and Rust code into the codebase – which is quite an undertaking […] and an incredibly tedious task to do it well."

Improvements

Review Processes
So the first thing I do is that a group of people would review every pull request exclusively from the view of security.
“I could always use more participants in the review process and so if I could hire some people, if I had the disposable income to do that, I would probably hire people to get more eyes on pull requests than just myself […]"

Improvements

Tooling
[W]ith unlimited resources, I would like some more investment into automatic tools that are better in like finding vulnerabilities and problems with code
“I think getting more tools and more CI-type tools to watch for that, because I think humans are vulnerable […] If I had unlimited budget and unlimited engineers, I’d really work on improving our testing systems."

Summary

Security Challenges
Guidance & Policies
Trust Processes
Improvements

Summary

  • Projects are highly diverse both in deployed security measures, trust processes, motivations
  • Growing scope and contributors → growing needs for security and trust processes
  • Smaller projects handle security and trust incidents "as they happen"

Open Source Supply Chain

Open Source code appears as foundation, glue, or during the build process in many software projects.

Closing

  • Against treating open source developers solely as data sources and review process blackboxes
  • For supporting open source projects in ways that better consider their individual strengths and limitations

Committed to Trust

Additional Slides


Interview Structure

Project Demographics

Project Structure

Release & Updates

  • Releases and updates based on direct community input and feedback
  • Exceptions from schedule for vulnerability fixes

Roles & Responsibilities

  • Projects have a variety of contributor hierarchies which are mostly relatively flat with two levels
  • Most of the projects do not staff teams dedicated to project security