Affected Functions: A Key to Understanding Open-Source Vulnerabilities

👋 This blog discusses the public availability of affected functions for open-source software vulnerabilities. If you’re interested in how vendors build automation to build private datasets expanding on publicly available data, let me know!

The software composition analysis (SCA) market has been flooded with vendors offering “reachability analysis,” a feature intended to reduce noise by determining if and when a function relevant to a given OSS package vulnerability is used (see the example below). But how do these vendors determine which functions introduce risk?

CVE-2024-0815
The advisory details a command injection vulnerability in the PaddlePaddle PyPI package versions up to and including 2.6.0. Specifically, the vulnerability affects one function: paddle.utils.download._wget_download().

Legacy SCA tools will check if your project uses one of these PaddlePaddle versions. In contrast, the reachability analysis feature enables SCA tools to verify that you’re using the paddle.utils.download._wget_download() function.

Publicly Available Sources

Firstly, I’d like to start by saying the industry is doing itself a disservice by keeping this data under wraps in proprietary databases, spending six or even seven figures annually to maintain and grow their datasets. Each vendor leverages their non-standard schema, and their data quality is often highly questionable.

👋 I’ve long said this would be a great business idea that can, and probably should, likely be bootstrapped. Selling this vulnerability data to vendors to empower reachability analysis —however, I’d much prefer this be a community effort that benefits everyone.

Today, some people trust the data (consumers who see reachability as a required feature), while others don’t (prefer to drown in endless false positives). Where are the people interested in measuring data quality and improving consumer confidence?

The simple truth is that vendors are unlikely to take this seriously unless the market speaks up and is willing to pay for it.

GitHub Advisory DB

GitHub maintains a vulnerability database that includes CVEs and GitHub-originated security advisories for open-source software. Advisories are stored as individual files and formatted in OSV Schema. Per the schema, affected functions should be noted under the affected[].ecosystem_specific field.

👋 An advisory may include details for more than one affected function or ecosystem.

Google’s OSV.dev

Like GitHub’s Advisory DB, Google maintains an open-source software security advisories database and ingests vulnerabilities from several sources, including GitHub. Typically, if I’m looking at language-package ecosystems like NPM and PyPI, I’ll go to GitHub, but Google’s OSV also contains vulnerabilities for open-source operating systems like Red Hat and Ubuntu. They also adhere to the OSV Schema and are who donated the OSV Schema to OpenSSF!

👋 counts on osv.dev will be greater because I removed malware advisories —none of these include affected functions.

*I noticed that OSV hasn’t effectively deduplicated GHSA and other databases (RustSec & GoVuln) causing duplicates e.g., CVE-2021-38191 returns the GitHub Security Advisory and the Rust Security Advisory.

Go Vulnerability DB

Are you wondering where GitHub and Google get these affected functions from? Some ecosystems, such as Go and Rust, capture these fields in their vulnerability reports. If you’re familiar with others, please let me know!

Rust Advisory DB

I suspect the actual count here is the same as osv.dev and GHSA; I decided it wasn’t worth calculating this better, given the repository stores advisories in markdown, which sometimes, but not always, uses some formatting. 🤮 

Open Discussions

OpenSSF OSV Schema: Unified field for describing affected functions

OpenJS: Create an affected function format specification for npm (JS) advisories

Conclusion

What does it say about our understanding of open-source software security when we only have visibility into affected functions for approximately 1% of security advisories? It's important to note that the data presented above only includes ecosystems with at least one affected function.

In contrast, the Go ecosystem boasts a commendable coverage rate of 31%. So, what accounts for this discrepancy? Although I have no data on this, I have found the affected symbols in the Go Vulnerability Database to be the most accurate and comprehensive—trust me, bro.

Vendors promoting reachability analysis assert that their tools can reduce findings by 85-98%. If true, it underscores the urgent need for better publicly available data on these advisories. Yet, these vendors' lack of transparency regarding their proprietary affected function datasets raises questions about the effectiveness of reachability analysis. Without independent data quality evaluations, we are left in the dark about how reliable these claims are.

Script Used For Counting

The following was used for GitHub Security Advisories, OSV, and the Go Vulnerability Database. Parsing the markdown files for the Rust Advisory Database was an ugly attempt, but I did adapt the following script to properly capture Rust Sec vulnerabilities that have been converted to osv schema and hosted on osv.dev.

import os
import json
from collections import defaultdict

def count_json_files_with_affected_functions_per_ecosystem(directory):
    ecosystem_files = defaultdict(list)  # Dictionary to store file names per ecosystem

    # Walk through the directory
    for root, _, files in os.walk(directory):
        for file in files:
            if file.endswith('.json'):
                file_path = os.path.join(root, file)
                try:
                    with open(file_path, 'r') as f:
                        data = json.load(f)
                        # Check if 'affected' exists in the JSON structure
                        if 'affected' in data:
                            for item in data['affected']:
                                if 'ecosystem_specific' in item:
                                    ecosystem_name = item['package']['ecosystem']
                                    
                                    # Check for affected_functions or functions
                                    affected_functions = item['ecosystem_specific'].get('affected_functions')
                                    functions = item['ecosystem_specific'].get('affects', {}).get('functions')
                                    imports = item['ecosystem_specific'].get('imports', [])

                                    # Check if symbols are present and non-empty
                                    symbols_present = any('symbols' in imp and imp['symbols'] for imp in imports)

                                    # Count the file if affected_functions is not None, functions is a non-empty list, or symbols are present
                                    if affected_functions is not None or (functions and isinstance(functions, list) and functions) or symbols_present:
                                        ecosystem_files[ecosystem_name].append(file_path)
                                        break  # No need to check further in this file
                except (json.JSONDecodeError, FileNotFoundError) as e:
                    print(f"Error reading {file_path}: {e}")

    return ecosystem_files

if __name__ == "__main__":
    current_directory = os.getcwd()  # Get the current working directory
    ecosystem_files = count_json_files_with_affected_functions_per_ecosystem(current_directory)
    
    print("JSON files containing 'affected_functions', non-empty 'functions', or 'symbols' per ecosystem:")
    for ecosystem, files in ecosystem_files.items():
        print(f"{ecosystem}: {len(files)} files")

Until Next Time! 👋

Hey, you made it to the bottom – thanks for sticking around!

Questions, ideas, or want to chat? Slide into my inbox! 💌

Don’t hesitate to forward if someone could benefit from this.

See you next Monday!
-Kyle

P.S. CramHacks has a Supporter tier! You can upgrade here to support CramHacks and its free weekly content 😃.