Seeking consensus on dh

Posted Jun 6, 2019 19:32 UTC (Thu) by martin.langhoff (guest, #61417)
Parent article: Seeking consensus on dh

I've worked with packaging in Debian and RH style. Since the advent of fedpkg (and other bits of consolidation of the rpmdev stack), the rpm side has gotten a couple orders of magnitude easier, and more consistent.

Because of that, a small number of maintainers can work through a large number of packages, and keep them mostly consistent. Maintainers can drop in on any package that's needing TLC and pretty much fix / update without breaking stuff. On Fedora, this happens all the time -- there's super-maintainers with rights over all packages.

It's like any project with strong consistent code style.

Debian packaging... well, it harkens back to the Perl era. TIMTOWDI. Some teams maintain a set of packages, and use consistent tools and style, achieving quality and leverage. Ubuntu folks had made some strides in their packages. But outside those oasis, it's a maze of hermits in caves, each maintaining their unique little toolchain. Super-maintainers like in Fedora would be impossible, and just wreak havoc.

True DDs love this balkanized world, and I love that they love it because I doubt anyone else does.

to post comments

Seeking consensus on dh

Posted Jun 6, 2019 20:13 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (15 responses)

I've been maintainig forks of the following Debian packages as part of a product which is now in "support only" mode: apache2, bind9, htmldoc, lilo, logrotate, ntpdate, openssl and postgres. Some of these are decidedly non-trivial (I'm still maintaining a libpcap fork for both Debian and Ubuntu). Hence, your statement that the Debian tooling would prevent people from working with packages they didn't create ("Super-maintainers like in Fedora would be impossible, and just wreak havoc. ") is obviously wrong.

Seeking consensus on dh

Posted Jun 6, 2019 20:56 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (13 responses)

> Hence, your statement that the Debian tooling would prevent people from working with packages they didn't create ("Super-maintainers like in Fedora would be impossible, and just wreak havoc. ") is obviously wrong.

No, his point goes beyond that. In Fedora is very common and typical to have non package maintainers commit across thousands of packages when there is an improvement in packaging that has been proposed and agree by Fedora packaging committee (and or Fedora Engineering steering committee if it is a more higher level change). Maintainers have the responsibilities in Fedora but not a tight or exclusive hold over the packages. In Debian, there is a strong association between packages and packagers and it is a different culture. Your forks of a bunch of packages that you maintain in-house has no impact on that cultural difference.

Seeking consensus on dh

Posted Jun 6, 2019 21:07 UTC (Thu) by rweikusat2 (subscriber, #117920) [Link] (12 responses)

> No, his point goes beyond that.

I wasn't addressing more than what I wrote about, namely, the supposed "impossibility" to work with packages one didn't create due to all of these different "styles": It's perfectly possible and - in fact - even common-place that "a small number of maintainers work with a large number of packages" in Debian, there's even a procedure for that called NMU (non-maintainer upload), usually employed for fixes which are considered urgent, IOW, security issues, these being done by a dedicated team of people.

Seeking consensus on dh

Posted Jun 7, 2019 0:58 UTC (Fri) by martin.langhoff (guest, #61417) [Link] (8 responses)

It's possible but the cognitive load is 10x due to diverse styles and tools.

It's a cultural thing.

When a codebase is managed by entrenched developers who refuse a common style, this is what you get.

Seeking consensus on dh

Posted Jun 7, 2019 14:49 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link] (7 responses)

This "cognitive load" negligent to non-existant. Either a package is trivial. Then, understanding it will be trivial as well. Or it's not. In that case, there'll be a lot of package-specific things which will necessarily differ, with the package building tools being a very small portion of that.

Seeking consensus on dh

Posted Jun 7, 2019 15:21 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

> In that case, there'll be a lot of package-specific things which will necessarily differ, with the package building tools being a very small portion of that.

With my over a decade of experience maintaining dozens and dozens of packages for an upstream distribution, consistency of package building tools matter a lot to people maintaining the distribution day in and day out. I wouldn't call that a small portion at all especially when you are bringing in new contributors.

Seeking consensus on dh

Posted Jun 7, 2019 15:34 UTC (Fri) by smurf (subscriber, #17840) [Link] (5 responses)

I disagree: there's more to packaging a distribution than touching a single package.

Suppose you want to fix a somewhat-trivial problem with your packaging like, for instance, replacing "/var/run" with "/run". Or, say, change the default flags of the compiler for more hardening. Or … well, whatever.

With "dh" this is a no-brainer. You update its defaults, rebuild the whole archive, and file bugs where that fails.
With trivial rpm specfile, this is also a no-brainer. More likely than not, there's a field for what you're trying to do, or a default value in RPM. Update it, rebuild the package, file a bug when that fails.

With anything nonstandard, you can't do that – your understanding does not translate well to a "sed" command, trivial package or not.
Even if the cognitive load was a mere ten seconds per package, you'd spend a whole week on something as big as Debian.

Packaging is not the only area where "diversity" ends up blocking progress. Source archives and packaging their debian/ subdirectories are another good example. Why do I still need to upload some Upstream tar archive plus a separately-packaged debian/ subdirectory with a debian/patches atrocity, when a git location plus commit ID would take up a whole load less bandwidthß

Seeking consensus on dh

Posted Jun 7, 2019 17:33 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link] (3 responses)

You may disagree with something but that's certainly not related to what I was writing about (namely, working with Debian packages doesn't become significantly more difficult because there's no enforced, standard auxiliary toolset for building one).

Seeking consensus on dh

Posted Jun 7, 2019 18:29 UTC (Fri) by smurf (subscriber, #17840) [Link] (2 responses)

Working with one or ten Debian package? no. Working with one or ten thousand? definitely.

It's fairly easy to automatically determine which build steps are "special" in a package if that package uses dh. Anything else? not so much.

Seeking consensus on dh

Posted Jun 7, 2019 21:05 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

Assuming that building a package takes 10s (unrealistic), someone doing nothing but "building packages" 24x7 would need more than a day to build "10,000 packages". A project of this size would necessarily need to be handled by a fairly large group of people. Also, there aren't "10,000" different toolchains for package building, the number is probably less than 10.

OTOH, feel free to believe whatever you want. I have a forked package I need to work on ...

Seeking consensus on dh

Posted Jun 7, 2019 21:40 UTC (Fri) by pizza (subscriber, #46) [Link]

> Assuming that building a package takes 10s (unrealistic), someone doing nothing but "building packages" 24x7 would need more than a day to build "10,000 packages".

Um, that's what automated build farms are for...?

Seeking consensus on dh

Posted Jun 10, 2019 19:52 UTC (Mon) by derobert (subscriber, #89569) [Link]

You're looking for dgit, which although it doesn't save the bandwidth (since the original + delta upload is still done), does save you from having to worry about it. Also, conceivably, if enough of Debian switches to it, getting rid of source packages (replacing them with git repositories) becomes possible.

There was also discussion about this recently on debian-devel, as Ian Jackson is trying to make sure it works with all git workflows, with the eventual goal of everyone using it.

Seeking consensus on dh

Posted Jun 7, 2019 3:58 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

> there's even a procedure for that called NMU (non-maintainer upload), usually employed for fixes which are considered urgent, IOW, security issues, these being done by a dedicated team of people.

Yep, you said it yourself. NMU is limited to urgent things and a small number of people. Fedora proven packagers are more in number and routinely make non urgent changes. Small targeted fixes are welcome and encouraged.

https://docs.fedoraproject.org/en-US/fesco/Provenpackager...

https://fedoraproject.org/wiki/Who_is_allowed_to_modify_w...

It is a huge difference in scope and general culture

Seeking consensus on dh

Posted Jun 7, 2019 14:52 UTC (Fri) by rweikusat2 (subscriber, #117920) [Link] (1 responses)

The original statement was about something being impossible. It's demonstrably not.

Seeking consensus on dh

Posted Jun 7, 2019 15:17 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

> The original statement was about something being impossible. It's demonstrably not.

Fair but it's also demonstrably not as easy or common as in Fedora

Seeking consensus on dh

Posted Jun 7, 2019 3:34 UTC (Fri) by martin.langhoff (guest, #61417) [Link]

To be clear here, it's not impossible, it's 10x harder, and a lot more brittle. And just not culturally acceptable in debian.

In Debian, an NMU is something people complain about. And with some reason -- the maintainer who did the NMU has a good chance of breaking stuff, because each package is unique in its packaging.

In Fedora, super maintainers work on hundreds or thousands of packages over a couple days regularly, and generally nothing breaks. NMUs are the rule, not the exception. Packages are cattle for them.