Who Writes the Bugs? A Deeper Look at 125,000 Kernel Vulnerabilities
pebblebed.com

・

63 points

・

MBCook

・

5 hours ago

19 comments

vlovich123 ・ an hour ago

> Half the kernel is still built by individuals: people using gmail.com, personal domains, or university emails. The "corporate takeover" narrative is overstated. Companies contribute heavily, but the kernel remains a genuinely collaborative project.

Isn't the assumption here flawed? Someone may be employed by a corporation but still use their gmail/personal domain/university domain. This needs to be cross-correlated against some secondary source of employment data to give a more accurate picture.

  • andyferris ・ 15 minutes ago

    I am a bit confused though here - should the kernel be written by unemployed people? Most contributors won’t have full-time funding.

vintagedave ・ 2 hours ago

This reads like Claude wrote it (more than ChatGPT.) Interesting data but I am unsure how actionable it is. Are they suggesting, for example, that specific commit messages get scanner more closely? Why is CAN more severe than Intel? (It does worry me. I feel like bugs, of any sort, in car systems are terrifying.)

  • aDyslecticCrow ・ 2 hours ago

    > CAN more severe than Intel

    I suspect the usage of the CAN driver in Linux is pretty low. The largest user of the Linux can driver is likely testing and diagnostics tooling for developing cars rather than the car themselves. Even when the car has a Linux computer, they often use multi CPU SOC's that run the real-time CAN logic separate from Linux, and only convey application logic into Linux.

    I could also speculate that the overlap between Linux kernel developers and automotive and industrial embedded systems is pretty low. So the high bug severity in the CAN driver could be developers contributing patches from a very different programming background?

    • myself248 ・ an hour ago

      > I could also speculate that the overlap between Linux kernel developers and automotive and industrial embedded systems is pretty low.

      Agreed.

      > So the high bug severity in the CAN driver could be developers contributing patches from a very different programming background?

      Background and situation. Their mindset is "I need this to work, right now", not "I need this to work, and not break anything else, forever".

    • earthscienceman ・ 44 minutes ago

      There are so many more embedded CAN systems beyond cars. Industrial battery management uses Linux and canbus, for example.

petterroea ・ 3 hours ago

Not happy with the lack of statistical testing, some of the smaller differences in % could probably be coincidence

dogleash ・ 4 hours ago

These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.

The map is not the territory.

  • palmotea ・ 3 hours ago

    > These smell like the kind of metrics that cause someone to feel informed and then to miss the forest for the trees. The kind of data for a "data driven" decision maker who will just invent a narrative to explain the numbers, and then do what they wanted to do all along.

    We need to increase reliability in the kernel, so the kernel team should fire the top 5 bug-introducers, to reduce the amount of bugs being introduced (https://pebblebed.com/blog/kernel-bugs-part2/05_author_analy...). Linus has got to go.

    • gchamonlive ・ 3 hours ago

      > We need to increase reliability in the kernel, so the kernel team should fire the top 5 bug-introducers, to reduce the amount of bugs being introduced (https://pebblebed.com/blog/kernel-bugs-part2/05_author_analy...). Linus has got to go.

      You've cut bugs being introduced while also reducing development costs by slashing team size. You deserve a promotion and an increase in equity.

  • alwa ・ 3 hours ago

    The LLM-tone doesn’t help:

    117 people meet this criteria. And the impact is dramatic:

    It’s strange to me to think of “bugfixes” in terms of a commodity. Different problem spaces between subsystems and thus different types of (and surfaces for) bugs; different contributor mixes; different number of eyes on them; different potential impacts…

    > CAN bus drivers top the list [of bug lifetime by subsystem]. These are used in automotive and industrial systems. Critical infrastructure with few maintainers watching.

    …or maybe higher-quality initial submissions, with most of the easy bugs already wrung out of them, so only subtle bugs remain (thus fewer to fix).

    Or adequately vigilant maintainers but low diversity of systems running that code, thus fewer users/situations where the bugs manifest, so they go unreported. Or poorer telemetry so an ordinary rate of latent bugs but they go undetected.

    Could be any, probably a little of all, can’t really tell from the analysis; and each cause would suggest a different response to improve quality.

kittikitti ・ 2 hours ago

I'm not sure why this isn't included in the blog, but I was curious about the ratio between bugs and commits. Presented here are my calculations in order of total number of bugs:

Intel : 11.86%

[1] Independent : 2.27%

Red Hat : 9.74%

Linaro : 12.73%

Google : 12.78%

AMD : 9.70%

The above is based on the bug count table in the article.

[1] I combined the total bug count for independent and kernel.org because they are combined for the total contributions here, https://github.com/quguanni/kernel-archaeology/blob/main/scr...

This suggests that corporations are introducing significantly more bugs than independent developers. However, I have not done statistical testing on this nor have I recreated the numbers. If I had to speculate, I would assume that the analysis from the author was partly vibe-coded or they purposely left this analysis out due to fear of retaliation. Extending my speculation would also include that corporations are purposely introducing bugs out of malice such that there are backdoors available for them. The author mentions that there is no "corporate takeover" but perhaps there are more interesting conclusions to be found.

  • rnewme ・ 2 hours ago

    What about the complexity and type of the committed code?

    • pixl97 ・ 2 hours ago

      Ya, a total guess on my part is the corporations are adding more things like complete drivers and kernel modules where individuals may be adding more smaller fixes.

  • PowerElectronix ・ an hour ago

    You can feel IC fear of being roasted by linus in those numbers.

charcircuit ・ 2 hours ago

I'd also like to see this broken down for C vs Rust.

jeffbee ・ 3 hours ago

Bugs Georg, who is an outlier and should be excluded from the analysis.

  • olivia-banks ・ 3 hours ago

    Strange how someone in a cave with no internet can push 10,000 bugs a day.