Software Bill of Materials (SBOM): The Gateway Drug to Supply Chain Security

TL;DR: In today’s world, SBOMs are littered with misinformation, almost always missing details, and are useless on their own.

If you’re reading this, you may be familiar with NTIA’s SBOM Myths vs. Facts, and if you trust everything you read on the internet, you will probably dislike this post. That’s okay ❤️.

I should probably note that I am not anti-SBOM. This article is specifically about my current issues with them. Maybe one day I’ll make a more positive post 🙂.

Software Bill of Materials (SBOM)

“A software bill of materials (SBOM) declares the inventory of components used to build a software artifact such as a software application.”

An SBOM is a detailed inventory of all components that comprise a piece of software. Just like a list of ingredients on a food package, an SBOM provides transparency, listing all open-source and proprietary elements, libraries, dependencies, and modules used by a project. It’s a manifest that includes version numbers, licenses, and other pertinent information about each component.

Everyone and their mother seems to have heard the term SBOM, especially in the last year, thanks to numerous regulations - and I’m partially grateful for this, given without it, most people wouldn’t have a clue what supply chain security is…

That said, the vast majority have been misguided into thinking that an SBOM on its own does anything for security. Or that it’s reasonably feasible even to generate a quality SBOM, let alone ingest and assess one.

With this in mind, take a second look at the definition of an SBOM. Where is the line drawn? Should it only include packages used by the application? Cloud Components? The developer’s IDE? Their operating system? The coffee beans they used that morning?

Well, there’s currently no definitive answer, so let’s stick with what’s likely the most feasible, which is simply the packages used by the application.

The Current State of SBOM Quality 🚽 

We’re in an era where creating an SBOM is more of a box-ticking exercise than a security measure. These documents are often incomplete, outdated, or simply inaccurate. It’s not just about having an inventory; it’s about having an accurate and actionable one.

Heck, we can’t even decide what format to use. Today, the two standouts are SDPX and CycloneDX. In my opinion, CycloneDX is the clear winner, but that’s not my choice to make.

I’m sure someone will point out the obvious: having an inventory of some is better than an inventory of none. Sure, whatever, good luck with that.

To back my claim, I’ll reference three major findings from A Viewpoint on Knowing Software: Bill of Materials Quality When You See It by Santiago Torres Arias, Dan Geer, and John Speed Meyers:

1. Just one percent of SBOMs meet the NTIA’s “minimum elements” criteria for all components.

2. Regardless of the tool or quality checked, most SBOMs provide abundant yet incomplete metadata.

3. Analyzing two SBOM tools on identical artifacts shows a discrepancy of over 10 components in 16% of cases, indicating that merely reviewing an SBOM isn’t sufficient to assess its quality.

There’s unfortunately no way to determine the total number of missed packages because there’s no source of truth. However, given the discrepancy between the two SBOM tools, we can reasonably say packages are being missed.

Lastly, even a complete SBOM does not necessarily translate to real identified vulnerabilities. In 2021, Contrast Security released its 2021 State of Open-source Security Report, which determined:

1. 62% of libraries found in applications are inactive—not used by the software in any way.

2. Even in active libraries, only 31% of classes are invoked, leaving 69% unused.

Considering the vast majority (>90%) of the project’s supply chain vulnerabilities are likely from transitive vulnerabilities (dependencies of dependencies), evaluating call paths to determine if your project uses the vulnerable component is critical.

Quality Checks

WooHoo! Resources are available to verify that you’re looking at a quality SBOM. But wait… how exactly does it know that?

Well, I heard you like checkboxes, so we gave you checkboxes that verify an SBOM is quality. What value does that provide? Well, it tells you that the tool used to generate it knew what is supposed to go into the SBOM - but no, it does nothing to determine if the information is accurate or complete.

That said, I’ll note some of the current projects anyhow because, at the very least, it’s nice to see that this issue is being thought about:

• NTIA: The Minimum Elements For a Software Bill of Materials (SBOM)

• NTIA: SPDX Checker

• CycloneDX: SBOM-Utility

• sbombenchmark.dev

• OWASP: BOM Maturity Model

I’m sure there are many more; these immediately came to mind.

You have an SBOM; now what? 🔚 

Odds are, if you have an SBOM, you are either working towards improving your internal security posture or you are conducting a risk evaluation of vendor-provided software.

This section contains less criticism since I feel it’s where an SBOM could eventually result in better security.

Internal Security

The challenge lies in turning these lists into actionable security insights. Dependency Track, a notable tool in this space, attempts to bridge this gap by tracking and analyzing the components listed in SBOMs. What’s crazy to me is that Dependency Track has been around for over 10 years.

“Before 2018, Dependency-Track likely had less than 10 organizations using it. Today, it has more than 10,000 organizations using it, in production, and is responsible for analyzing over 300M components every month, minimum.”

For organizations with a large number of projects, the task of managing and updating SBOMs becomes increasingly complex and resource-intensive. The logistics of continuously monitoring each repository for new vulnerabilities, updating dependencies, and ensuring compliance with security standards can quickly become unmanageable.

Vendor Risk

In vendor risk assessment, SBOMs are often seen as a tool for evaluating third-party software security. However, as with internal security, their effectiveness is hampered by the same issues of inaccuracy and lack of standardization. A vendor might provide an SBOM generated using a tool that misses several key dependencies or fails to capture the latest updates, leading to underestimating potential risks.

The absence of a universal standard for SBOMs exacerbates this problem. Vendors might use different formats or levels of detail, making conducting a fair and thorough assessment difficult. In some cases, vendors might even prefer using tools that generate less detailed SBOMs, as these are less likely to reveal vulnerabilities in their software.

SBOM Vulnerabilities

This is where things really fall apart. Should an SBOM include vulnerabilities? Regardless of whether vulnerabilities are stored statically in the SBOM or generated based on the SBOM, you run into the same issues.

What’s the expected vulnerability database to use to determine affected packages? If you solely look at CVEs - as most do - forked packages will always have zero vulnerabilities 🤔.

More importantly, it has been proven repeatedly that identifying vulnerabilities solely based on whether you use it is ineffective. The research paper, A Comparative Study of Vulnerability Reporting by Software Composition Analysis Tools, found only 2.1% of roughly 2,500 vulnerability alerts to be reachable through static code analysis. This means that you can reduce false positives by up to 98% via code-scanning reachability. But that is not feasible with only an SBOM.

Conclusion

So, where does this leave us with SBOMs? To put it bluntly, we’re miles away from where we need to be. The hype around SBOMs has been loud and clear. Still, the reality is a cacophony of misinformation, inconsistencies, and a massive underestimation of the logistical nightmare of managing them in large-scale environments.

As for vendor risk, it’s like navigating a minefield blindfolded. Every vendor SBOM is a gamble – some are goldmines of information, and others are practically useless. Without standardization or a reliable means to verify their completeness, SBOMs in their current form are akin to a security placebo – they offer the illusion of safety but little in the way of real protection.

Then there’s the vulnerability problem. Relying solely on CVEs is a shot in the dark, and the lack of a universal standard for vulnerability reporting, especially for transitive vulnerabilities, only adds to the chaos. The research paper highlighting the minuscule percentage of reachable vulnerabilities through static code analysis is a wake-up call. It’s a stark reminder that a list of ingredients (or components, in this case) is meaningless if you don’t know what to do with them.

In essence, SBOMs, as they stand today, are a half-baked solution to a complex problem. They’re a step in the right direction, but without significant improvements in accuracy, standardization, and practical usability, they’ll remain more of a buzzword than a boon in the world of cybersecurity.