Supplementing CVEs with !CVEs

Posted Dec 8, 2023 13:00 UTC (Fri) by farnz (subscriber, #17727)
In reply to: Supplementing CVEs with !CVEs by pombredanne
Parent article: Supplementing CVEs with !CVEs

The core problem is social; the idea behind the CVE system is that there should be one database of severe bugs, such that if you are vulnerable to any of the bugs in the database, you know that you have a big problem to fix. This then allows a secondary thing where bugs in commonly shared code can be referred to by CVE number, so that (for example) a bug in the EDG language front-ends can be referred to by Microsoft, Intel, SGI and other downstream users with the same CVE number, rather than needing one bug reference per compiler vendor who used the EDG front-end.

This, in turn, was beneficial to consumers, since instead of getting (say) three different bug reports from Microsoft, Intel and SGI about a compiler bug, I got three different bug reports all containing the same CVE string, so I knew that I had a common bug here, and not a case where I had three different bugs to deal with. This was especially important in the days when lots of software was both proprietary and white-labelled, so I'd get bug reports about an issue in (making names up now) HP-UX Secure Transport Services, Sun Transport Layer Security System and SGI Secured Communication Layer, all of which would turn out to be the same OpenSSL bug (because the products were actually just rebrandings of OpenSSL).

However, once you have a big shared database of "severe" bugs, you tempt people to build metrics around it; and when people build metrics, Goodhart's law follows close behind. In the case of CVEs, the big problem is that people use "number of CVEs you found" as a metric for how good someone is at finding severe bugs; this leads to people trying to get many many CVEs in their name, in order to win at the metric, making it useless for its original purpose (since it's now a metric for how good you are at finding a CNA who'll let you get a CVE number for a report).

And that social problem is hard - we need people to stop caring about CVEs per-se, and to start caring about only the subset of CVEs that represent "real" issues (FSVO of "real" and issues). Trouble is that the vendors have a reason to downplay the severity of every issue, so you need a trustworthy source of information - how do you distinguish Daniel Stenberg's explanation of why a vulnerability should have scored 3.3 not the original 9.8 from a motivated closed-source vendor lying about the severity to get the vulnerability score down, absent technical ability?

to post comments

Supplementing CVEs with !CVEs

Posted Dec 8, 2023 14:11 UTC (Fri) by atnot (guest, #124910) [Link] (1 responses)

> the idea behind the CVE system is that there should be one database of severe bugs, such that if you are vulnerable to any of the bugs in the database, you know that you have a big problem to fix. This then allows a secondary thing where bugs in commonly shared code can be referred to by CVE number

To my knowledge this is backwards, at least historically. The issue of finding out whether your version was vulnerable was already a solved issue: you just asked your vendor, which had their own bug tracker ID. People weren't really pulling in random versions of hundreds of dependencies to the degree they are now, this sort of scanning wasn't really that necessary.

So the primary purpose was really more to enable more cooperation for security bugs by having an independent central place that would host issue descriptions and give them a unique number. You can sort of see this from the process, I think: CVEs are assigned mostly by affected companies, they are not really given more than cursory verification and were originally just a straightforward republishing of the authors description, without any further classification by MITRE. Attaching meaning to those IDs was the job of other entities, like the affected companies and national CERTs. In this regard, CVEs still work as intended.

I think where this started going wrong was with the introduction of things like CVSS, which increased their duties from merely hosting a database of claims to interpreting those claims, a thing which they are really wholly unequiped for doing. There were plenty of other causes of course, but all of this moves the CVD from a role of merely neutrally numbering important claims to being a factual, accurate description of every vulnerability ever discovered, which the process is not remotely set up for.

Supplementing CVEs with !CVEs

Posted Dec 18, 2023 15:06 UTC (Mon) by nim-nim (subscriber, #34454) [Link]

It started going wrong when tooling and free software made it dirt cheap and easy to pull in other people’s software. For a time all was dandy. And then the suits discovered that the average dev had no intention whatsoever of keeping up with third party code updates and security vulnerabilities (helloo log4j, brought to you by the Apache Open Source way: free-as-a-beer-code, no due diligence nor obligation, just don’t get caught by security scanners).

So now no one trusts whatever the dev vendor says.