Ransomware dominated the security discourse two years ago and now security threat models evolved to consider threats that come from within: code which is imported into the organization. Before 2020 most companies could get by with a fortified perimeter to keep attackers out. Now successful attacks are increasingly launched by compromising open source dependencies, prompt injections, or other exploits which target systems and processes that jump back and forth between the internal/external network boundaries. The flashy and sophisticated breaches that make the news obscure the mundane every-day risks that live in our CI systems, developer laptops, and technical debt.
I have thought a lot about the dependency tree and security for a long time, mostly influenced from my time as the Infrastructure Officer and board member of the Jenkins project. Jenkins has a robust plugin ecosystem but the security of that ecosystem affects a lot more than just the individual plugins, one bad plugin could compromise an entire installation, which made it a key focus for us. Many of the folks I worked with went on to help found Chainguard which is not coincidental!
For most small organizations they have fairly limited capacity to address the sort of sneaky hidden security issues lurking in their dependency chain, so below is a series of thoughts I wanted to share for smaller teams to frame their security thinking.
Build Environment security
GitHub Actions for most organizations has a very weak security posture. While I might perhaps recommend running a more controlled Jenkins environment inside of a corporate network perimeter, for many small teams GitHub Actions is too easy and too free to avoid. GitHub Actions can be made a lot more secure by trading some convenience for friction. Here are some settings changes to consider.
Disallow all actions should be on by default. Right now the GitHub default for Actions means that any person/agent can reference any GitHubaction anywhere and that action will run with our organizational secrets. This is a barn door that has been left open and must be closed immediately.
SHA1 action pinning is a newer
feature
that is less used generally speaking because it's tedious. This does prevent
tag/ref poisoning where an attacker might re-push a compromised workflow's v3
tag with a malicious ref. This is a good backlog task, but not something
practical to immediately apply. The restrictions above are a more practical
first step with less dramatic impact.
The big thing I don't like about SHA1 pinning is that it makes maintenance of
action versions lifecycle much more tedious. What is probably a better path
forward if we want to be the most secure is to only Allow enterprise actions
and then just privately fork upstream actions into a bespoke org for this purpose in the
enterprise, e.g. buoyant-actions.
Runners
There does not seem to be any way in GitHub to restrict the use of GitHub-hosted runners. Without this control it's difficult to ensure that actions run in the organization are run in the appropriately sandboxed or controlled environment. Spinning up Runners inside of locked down network environment with limited internet egress is a great security measure but it can be difficult to keep teams from inadvertently falling back to GitHub-hosted runners.
Using only corporate-hosted runners means you can apply configuration defaults which only allow access to third party resources that are trusted, reducing data exfiltration or compromise risk.
Ideally all package resolutions happen through a registry proxy like JFrog's
Artifactory, which does carry some cost
implications with it, but using a single choke point has innumerable security
benefits. This works best when the use of upstream npm, pypi, etc repositories can be blocked on the runners themselves.
Secrets
Organization-level secrets are convenient but should not be used if avoidable. Like many other aspects of GitHub Actions the convenient-defaults also act as security issues. Only using repo-level secrets limits the blast radius of a compromise and encourages the use of least-privilege workload-specific credentials.
Secure Dependency Management
Building from a foundation of strong network isolation the cheapest/easiest/fastest way to apply some secure practices are by restricting public registries wholesale from build environments, whether that is through CI or not.
I have seen some enterprises go so far as to push network management policies to developer laptops which also restricts the use of public source repositories in preference to internally governed ones like Artifactory.
This can reduce the impact for supply chain attacks and also reduce the likelihood of accidental IP disclosure to these public registries.
The recent disclosure of Claude Code source maps would have been prevented if every system involved at Anthropic was required to use their internal package registry as opposed to allowing developers/others to just yolo to npmo.
The minimumReleaseAge which exists in NPM can be applied at the Artifactory
level rather than needing to chase the long tail of package formats and build
systems.
The proxy cache can become a single source of failure, which means investment in a robust enough system to manage the thundering hoards of CI workloads is important. The configuration in a system like Artifactory can also be a little tricky to ensure is correct before applying policies.
The cache can also serve as a place to blocklist packages as part of the #security patch lifecycle management practices. In other organizations I have seen them apply policies which actively prevent known insecure packages from being downloaded. For example if 1.1.3 has a known RCE, the proxy can make it such that 1.1.3 is effectively impossible to install within the organization
Reproducible Builds
Reproducible builds have been a pretty hot topic in the release engineering domain for the past 5-10 years but we still haven't seen much inroads in the tier of company that we're at.
The closest thing to a "reproducible build" most teams have is "we check in our uv.lock" which doesn't really do much, especially when hitting public package registries.
Software Bill of Materials (SBOMs) are a better way to approach this and can be incorporated into container build processes to provide artifact provenance.
Frankly most organizations are unwilling to invest the time in introducing security at the build level in this way. The returns start to diminish rapidly for many small teams, and so long as a package proxy is in front of the builds, that is typically "good enough."
Another route for more reproducible builds is to simply vendor dependencies into the source tree, but that is a fairly contentions topic for another day!
We all benefit from a robust and secure open source ecosystem but software security requires trade-off decisions which are unique to each organization.
Fundamentally it is impossible to "secure" software that comes from somewhere else. It is the responsibility of the team pulling the software to ensure they are utilizing source in a responsible way. Allowing the arbitrary retrieval of code from the internet is the key problem that I think most small organizations should address when news of package compromises reverberates through the industry.
There is no such thing as 100% secure but there are some immediate changes you can make to improve your defensive posture. Strive for more secure than yesterday.
Building a secure data infrastructure requires a layered approach to both the infrastructure tself and software delivery lifecycle. If your organization would like to talk more about how we can build more confidence into your platform, drop me an email and we'll chat!!
