Comparing SLSA to the CNCF Software Supply Chain Best Practices Recommendations

This is a little bit of a different post for “Read the Source” in that the source we are reading here isn’t “code” per se.

There has been a lot of discussion lately of supply chain security (and with good reason!), and two efforts in particular have drawn my attention. The first is the Cloud Native Computing Foundation’s “Software Supply Chain Best Practices” paper, which I helped to write and edit. The second is the SLSA project, originally by Google and now under the auspices of the OpenSSF.

CNCF laid out a series of 49 recommendations covering five areas of supply chain security: Source Code, Materials (ie, dependencies), Build Pipelines, Artefacts, and Deployments. They categorized those requirements as being for “moderate/high” security and risk levels or for “high” security and risk levels. In other words, the stuff anyone trying to be reasonably secure should be doing, and the stuff that really high security operations should be doing.

SLSA presents four levels of assurance focused primarily on the question of provenance. In other words, its focus is on answering the question “how do I know this is the package I think it is?”, which is an important component of supply chain security. The levels increase in strictness and the level of assurance that they provide.

I thought it would interesting to compare the two frameworks. My approach was to map all of the CNCF recommendations to the SLSA levels. It turns out most of them don’t really map, but that’s not surprising because CNCF is focused on a larger set of security concerns than provenance. So to be fair to the SLSA team, in the chart below I’ve highlighted the CNCF recommendations that (a) aren’t covered by SLSA, AND (b) seem to have some relevance to the question of provenance. In other words, the CNCF recommendations that aren’t mapped to SLSA but are also focused on other security concerns besides provenance are in this chart, but aren’t highlighted.

Here’s the chart:

You can also download it as a .pdf

And here are a few observations:

1. When compared side-by-side, it is very clear how different the scopes of these projects are. SLSA is overwhelmingly focused on provenance: collecting the metadata required to verify that a package is what it claims to be. But this doesn’t do anything to address the contents of that package: is it malicious? Does it contain vulnerabilities? What about its dependencies? To some extent, we can never have 100% assurance about these questions, but CNCF includes a lot of recommendations that increase our confidence around those other security concerns.
2. The place where Provenance and Security mostly clearly overlap is in the build pipeline. This makes sense: the question provenance is trying to answer is whether the package is what it says on the tin. The place with the most surface area for this to change is in between the source code and the published package, so most of SLSA’s recommendations are focused on ensuring that this build pipeline doesn’t introduce unexpected changes.
3. SLSA has very little to say about source code and dependencies. To be fair, most of the CNCF recommendations regarding source code and dependencies have more to do with checking the contents of those items than with checking their provenance. So they are at least somewhat out of SLSA’s bailiwick. However, even the provenance relevant recommendations around these items get little love from SLSA. In other words, SLSA seems to assume that our concern about provenance mostly starts when a build is triggered, but that we aren’t especially worried about whether someone has tampered with our source code itself or with the dependencies we are ingesting. Higher levels of SLSA do take some steps to address source code security (ie, requiring MFA for contributors and protecting commit history). But these are not required at lower levels of SLSA, and other considerations (like signing commits, which provides some provenance about where those commits are coming from) aren’t mentioned anywhere in the SLSA documentation. Additionally, SLSA can be applied recursively to dependencies, but, interestingly, the framework explicitly states that SLSA levels are not “transitive”, meaning this recursive analysis doesn’t actually affect the top-level project’s SLSA score.
4. SLSA doesn’t address securing deployments. I suspect this reflects an assumption that if the provenance data has been produced as specified, it will be available to a consumer for verifying the artefact they are deploying. However, SLSA doesn’t explicitly require that this be the case, and that might be problematic (especially around closed source software).

It’s important to note that SLSA is still in its very early stages. In other words, none of the above observations should be taken as written in stone because it’s all subject to change. These observations apply to the current state of the SLSA framework as of mid-August 2021. I’m also happy to hear corrections to my reading of the SLSA requirements if I’ve misinterpreted anything here. If you convince me that I have, I’ll republish this with updated information.

I appreciate what the SLSA project is doing. Supply Chain security contains a number of sub-problems, some of which are inherently intractable. As with much of security, we can only say “this is secure as far as we know.” However, at least one aspect of supply chain security can be cryptographically guaranteed: the “provenance” of a package, or its definitive claim to have arrived to us untampered with so that it contains exactly what it says it contains. This has a lot of important security implications. It addresses one large area of supply chain concern: interference in the build or distribution process. Additionally, and similarly to the promise of an SBOM, provenance allows us to know with certainty what components of a system are subject to a newly identified vulnerability (because we can track with certainty which components contain the vulnerable code). SLSA provides a stair-stepped approach to achieving this sort of provenance. However, it comes with two large caveats that need to be kept in mind by people in the industry who are examining (and potentially relying on) it: first, that provenance is not the same thing as security. Just because you know that a package is what it says it is, doesn’t mean you should trust what’s in it. And second, that there are still some growing edges to the model of provenance that SLSA provides, and those may yet undermine the assurances that it gives us. It’s a good start, but there is still more work to be done.