Linux kernel 'getting buggier,' leader says (ZDNet)

[Posted May 8, 2006 by corbet]

ZDNet reports from Andrew Morton's talk at LinuxTag. "Morton said he hasn't yet proved this statistically, but has noticed that he is getting more e-mails with bug reports. If he is able to confirm the increasing defect rate, he may temporarily halt the kernel development process to spend time resolving issues."

to post comments

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 14:44 UTC (Mon) by miah (guest, #639) [Link] (11 responses)

Maybe its just that more people are actually reporting bugs now?

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 15:38 UTC (Mon) by kirkengaard (guest, #15022) [Link] (1 responses)

It is possible that feedback is simply becoming more responsive, but that's a social change which is harder to quantify.

It seems more likely, prima facie, that 'more bug reports' is due to 'more bugs', simply because that conforms more to previously established realities. Codebases with high churn tend to develop bugs in proportion to the churn. Feedback does not tend to become more responsive without external pressure; feedback doesn't even tend to be *useful* without serious training in how to report bugs. New developers do tend to be less versed in existing protocol. Large changes to core functionality do tend to make old code break unpredictably.

Occam's razor tends to lean towards bugs in code as opposed to better human response.

And, have a look at bugzilla. It doesn't seem like we're at some sort of peak.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 9, 2006 8:17 UTC (Tue) by sepreece (guest, #19270) [Link]

I read his complaint as based more on what he saw than on what was reported - e.g., the comment about the proportion of patches that don't even compile when he gets them...

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 15:49 UTC (Mon) by quartz (guest, #37351) [Link] (8 responses)

Or simply that the number of people using the software has increased substantially lately? Lots of government agencies trying it out, IBM, Novell and others putting lots of money into testing, maybe?

So many possible correlations, it's unfortunate someone with as much visibility as Mr. Morton chose to write those words not too carefully (you're not paranoid if they're really watching :)

[]s Gus

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 15:57 UTC (Mon) by kirkengaard (guest, #15022) [Link] (6 responses)

" So many possible correlations, it's unfortunate someone with as much visibility as Mr. Morton chose to write those words not too carefully (you're not paranoid if they're really watching :)"

We do not hide. What part of "open" don't you get?

Mitigation

Posted May 8, 2006 16:07 UTC (Mon) by kirkengaard (guest, #15022) [Link] (5 responses)

Less bluntly: the community, and Linux in general, has nothing whatever to fear from good negative feedback. Good negative feedback should induce constructive activity. Andrew's comment was properly hedged (he admitted he hadn't quantified the exact nature of the problem), and proposed properly constructive solutions.

Shoot your canary and you never know when you'll die in the mineshaft.

Mitigation

Posted May 8, 2006 16:21 UTC (Mon) by tjc (guest, #137) [Link] (4 responses)

It's the "The sky is falling, Linux development is halting" articles from the mainstream press (not to mention any names like John Dvorak) that will be unappreciated by most of us.

Mitigation

Posted May 8, 2006 16:31 UTC (Mon) by kirkengaard (guest, #15022) [Link] (1 responses)

I appreciate the articles and their authors as well as the rest of you; have you a better way of stopping them than shooting the canary and compromising the develoment process?

You can't control what the media picks up. You want to make Andrew more media-potable? How about Linus? How long do any of these stories actually last? What you cannot control, ignore. Focus on what is actually important.

Besides, there seems to be a competent vulture-shooting squad. Save the canaries, let them shoot the vultures. :)

Mitigation

Posted May 9, 2006 1:02 UTC (Tue) by grouch (guest, #27289) [Link]

"You want to make Andrew more media-potable? How about Linus?"

That's actually slightly scary. Can you imagine either of them bubbling about 'architecting people-enabling featuresets on a going-forward basis'?

I like coders who just tell it like it is; makes me more confident in using their work. :)

Halting

Posted May 8, 2006 17:02 UTC (Mon) by AnswerGuy (guest, #1256) [Link] (1 responses)

Any journalist who would write such "Chicken Little" articles over the prospect of a *temporary deferral* of new feature merges (a "halting of development") would find his or her credibility among technical readers to be severely undermined.

Even many non-technical readers (those with skills in critical analysis -- such as decision makers in the Fortune 500 and Fortune 1000) would see through such sensationalism.

For Andrew Morton to temporarily block inclusion of all patches other than well tested and minimal fixes might be just the thing to encourage more of our developer base to scan through the bits of code with which they are familiar and do some serious janitorial work.

I know there is also a Kernel Janitors project which might benefit from a little more mentoring and attention.

Perhaps declaring one month to be "code cleaning" or "spring cleaning" would be sufficient (with the possibility of extending it by another month if things go so slowly).

JimD

decision makers...

Posted May 9, 2006 17:04 UTC (Tue) by pflugstad (subscriber, #224) [Link]

Even many non-technical readers (those with skills in critical analysis -- such as decision makers in the Fortune 500 and Fortune 1000) would see through such sensationalism.

Just my opinion/experience, but I think you're drastically overstating the abilities of such "decision makers".

Which means that what the "journalist" writes does have an impact. Just look at the continuing air time people like John "Let's whack the /. hornets nest" Dvorak and Rob "I am BillG's love child" Enderle keep getting. (Aside: if the /. editors had sense, they'd permanently ban any Dvorak article - the guy is clearly just a troll at this point).

You may not listen to them (having correctly concluded they are idiots), but "decision makers", in the absense of additional information, probably will. The only real solution seems to be to put out the right message, continually and constantly. This is slowly happening as companies like IBM engage their marketing arms constructively.

I think my point is that stories like this don't help the perception of Linux/OSS. They need to be clarified for all the "decision makers" out there who only see the sensationalist part.

Should we change Linux/Andrew - absolutely not. But there needs to be feedback and clarification, both to the journalist, their editors and "decision makers" about what the real story is.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 16:14 UTC (Mon) by kirkengaard (guest, #15022) [Link]

You do, however, have a good point which may change the shave of Occam's razor. Not only are there more developers, there are also more testing facilities, of potentially higher caliber and wider architecture variety.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 16:00 UTC (Mon) by kirkengaard (guest, #15022) [Link]

"A little action item I've given myself is to confirm that this increasing defect rate is really happening," he said. "If it is, we need to do something about it."

"Kernel developers will need to reapportion their time and spend more time fixing bugs," he added. "We may possibly have a bug fix-only kernel cycle, which is purely for fixing up long-standing bugs."

There is, for the -rc process, a theoretical "no big new patches" point, and that is post-rc1. After that point, bugs noted in the -rc get fixed, and smaller patches come in. This doesn't seem to have ever been thoroughly "bug-fixes-only", as 2.6.x.y tend to be, but the mainline has a stabilizing process.

Andrew is in a good position to see the bug problems; notice how much less frequent the -mm releases have become in the last few -rc's. I quote from 2.6.17-rc2-mm1:

- It took six hours work to get this release building and linking in just a
basic fashion on eight-odd architectures. It's getting out of control.

He has to make his zoo of patches apply workably to the point-release or -rc. While we have a stable group for mainline, there isn't the same kind of attention paid to the -mm branch.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 16:42 UTC (Mon) by cbcbcb (subscriber, #10350) [Link] (4 responses)

this is what all the posts saying "please open 2.7" have been trying to communicate all along.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 18:40 UTC (Mon) by philips (guest, #937) [Link] (3 responses)

Or side-effect of new versioning system. After all, it was introduced to "ease" introduction of new features - not stability.

But on other note, I appreciate such news. Now we now we all know for sure what for Red Hat uses its bug tracking system - to count bugs. Before, some silly users thought that somebody over there was actually paying attention to the bugs themselves.

But 2.7 is open

Posted May 9, 2006 2:36 UTC (Tue) by JoeBuck (guest, #2330) [Link] (1 responses)

The unstable development tree is 2.6; new features and API breakage happen with every release. The stable kernels are the ones with four numbers (and a large last number), or distributor kernels where the distributor froze a while back and backports security fixes.

There really isn't anything that could go into 2.7 that isn't going in to 2.6-latest right now.

But 2.7 is open

Posted May 13, 2006 23:50 UTC (Sat) by BackSeat (guest, #1886) [Link]

There really isn't anything that could go into 2.7 that isn't going in to 2.6-latest right now.

Exactly. Which is why we need a 2.7. Using the "stable" branch for new features, algorithm changes and bug-shaking isn't what users want. I could accept that the traditional development model based around odd/even minor version numbers might not work for the developers, but the 2.6 development model has only shifted the dissatisfaction to the consumers: it's hard to see that as a desirable outcome.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 10, 2006 17:58 UTC (Wed) by davej (subscriber, #354) [Link]

Yes. I spend my whole day, doing nothing else, but counting bugs, and eating cheetos.

Seriously, if this were the case, the situation would look a lot worse than it actually does. The reasoning behind maintaining statistics is to so we can get some idea of how good/bad the situation is, and whether or not we're actually making progress.

Linux kernel 'getting buggier,' leader says (ZDNet)

Posted May 8, 2006 16:45 UTC (Mon) by iabervon (subscriber, #722) [Link]

Is there actually a primary source available for this? Not that I don't trust ZDNet, but going through news articles makes it unclear whether the issue is more than that people are sending him patches that they haven't tested with their feature turned off.

Dave Jones' blog entry

Posted May 8, 2006 17:21 UTC (Mon) by abatters (✭ supporter ✭, #6932) [Link]

Here is Dave Jones' response to the article.