Systemd catches up with bind events

Posted Nov 16, 2020 0:20 UTC (Mon) by nevets (subscriber, #11875)
In reply to: Systemd catches up with bind events by pbonzini
Parent article: Systemd catches up with bind events

It wasn't really the command line usage that was the problem. The real problem was that the command line triggered systemd to spam the kernel printk buffer so much that we lost all debug messages from the kernel. When reporting this as a problem, instead of saying there was a bug in systemd, we were told it was not a bug and systemd is perfectly fine using the debug command line to trigger writing debug messages in the kernel printk buffer. I thought the real solution was to separate kernel writes into printk from userspace writes, preventing userspace from overwriting what the kernel produces.

This would not have escalated the way it did if we were told from the beginning, "oh there's a bug in systemd that causes it to spam the buffer, please upgrade to a fixed version". But instead told to bugger off. Yes, it really is a lack of communication and good faith between the two communities and I hope we can work better in the future.

to post comments

Systemd catches up with bind events

Posted Nov 16, 2020 7:55 UTC (Mon) by pbonzini (subscriber, #60935) [Link]

> oh there's a bug in systemd that causes it to spam the buffer, please upgrade to a fixed version

Technically the upstream people couldn't have known, since the bug was introduced by an incorrect distro backport. And if a buggy systemd, one that spews assertion failures all the time, will slow boot down to a crawl, the systemd people might even consider that to be a feature. It can and will happen for kernel WARNs as well, and a buggy PID 1 is not much better than a buggy kernel. But these are details, and in general I think we agree.

What this shows to me, is that Linux is sorely lacking postmortems. Whenever Linus screams at me, I try to figure out what went wrong in my workflow and how I can improve it to avoid being screamed at in the future. On the other hand, if 5 years later people still believe that "debug" is a sacred part of the kernel command line (and not the more nuanced explanation that you gave), something went wrong on the kernel side in figuring out what happened.