Seeking consensus on dh

Debian takes an almost completely "hands off" approach to the decisions that Debian developers (DDs) can make in regard to the packaging and maintenance of their packages. That leads to maximal freedom for DDs, but impacts the project in other ways, some of which may be less than entirely desirable. New Debian project leader (DPL) Sam Hartman started a conversation about potential changes to the Debian packaging requirements back in mid-May. In something of a departure from the Debian tradition of nearly endless discussion without reaching a conclusion (and, possibly, punting the decision to the technical committee or a vote in a general resolution), Hartman has instead tried to guide the discussion toward reaching some kind of rough consensus.

The question revolves around an adjunct to the debhelper tool that is used to build many Debian packages. The additional tool is a "command sequencer" for debhelper commands; it is called dh. Debhelper has commands that get invoked from the rules file that is used to build a .deb from the source code and other files that are part of a Debian package. By default, dh steps through a sequence of debhelper commands that should suffice to build many types of packages; if some of the steps need overrides or changes, that can be handled as well. In effect, dh encapsulates the standard way to build a Debian package using debhelper.

But not all packages use dh, so Hartman asked whether the distribution wanted to require, or at least recommend, the use of dh. In that posting to debian-devel, he noted that some have said that a package not using dh has a "package smell", which is an indication that the maintainers should consider fixing it. His question might ultimately boil down to "whether maintainers should be expected to apply well-written patches to convert a package to using dh".

Hartman noted that there will likely always be exceptions for some packages where using dh does not make sense. He also summarized some of the reasons for and against the idea. Using "dh makes packaging simpler", he said, but it is more than just that. It also makes it easier for other maintainers to understand and help out with package maintenance. That can make a real difference when there are tree-wide efforts, like hardening or reproducible builds, but it also would tend to draw DDs together as a team. The main argument against a change toward pushing dh is that it would introduce bugs in package building simply due to conversion mistakes. Beyond that:

Holger Levsen thought that the Debian Policy Manual should be changed to have dh as a "should" requirement, except in two specific areas: those packages that already use the Common Debian Build System (CDBS) and those packages that debhelper is dependent upon. Eventually, the "should" could become a "must". The Haskell ecosystem uses features of CDBS (which is seemingly not documented anywhere) that are not available in debhelper, so it may not be a good candidate for moving to dh, at least at this point. There has been some work to add a dh_haskell command to debhelper, Sean Whitton said, but it has stalled. Marc Dequènes pointed out that CDBS may be on its way toward retirement at this point, which should be kept in mind. Others agreed with Levsen's ideas, though some would like to see debhelper eventually take over for CDBS, which fits well if CDBS is on its way out.

But Marco d'Itri wondered why he would convert his debhelper-using packages to dh: "I use debhelper in all of my packages but I have never switched to dh: why should I bother? 'Everybody is doing this' is not much of an argument." Simon McVittie had a lengthy response that described some of the extras that dh provides. Essentially, it will help those who modify the packages ("perhaps your future self") by allowing them to ignore various semi-tedious details that have to be worked out without it. In addition, those details can change over time, which will not necessarily be reflected in packages that only use debhelper.

There was also discussion of whether it was a appropriate to use the non-maintainer upload (NMU) process to change a package to using dh. In general, it was not seen as a reasonable way to switch a package to dh. As Scott Kitterman put it: "A likely bug inducing drive-by NMU is not helpful." He thinks that new packages should, in general, be built using dh, but is less convinced that existing packages will truly benefit—and the likely result will be lots of tricky bugs:

Whitton largely agreed with Kitterman about the distinction between new and existing packages. Ian Jackson also thought it made sense, though he did have an anecdotal data point about his efforts to convert the Xen package, which worked out well, overall. For policy, though, he didn't think converting should be mandated:

As the conversation started winding down a bit, Hartman said that he planned to try to summarize where consensus was found and not found in the discussion. That resulted in his consensus call post on May 25. He used the consensus process in RFC 7282 as a model, though, of course, it "has no force here"; it does, however, have some useful thoughts, he said:

He asked that people make comments by June 16 and to clearly distinguish where they thought his summary was not accurate versus comments meant to help establish a different consensus. His summary laid out some of the issues that recurred in the earlier thread: that dh makes cross-archive changes easier, that converting to dh could be seen as churn that made other people's efforts harder, especially if it wasn't tested well, and that Debian has "valued respecting the maintainer's preference above most other factors". Pushing for more uniformity is perhaps a step away from that last item.

He also summarized the areas where he thought rough consensus had been reached. One is that, except in certain circumstances, using dh is the right thing for a maintainer to do. Those certain circumstances were outlined (e.g. the Haskell ecosystem), though he called them "exceptional circumstances", which Jackson thought sent a slightly wrong message. Jackson suggested "unusual circumstances" since that does not necessarily imply rarity, saying that the consensus he heard was "that dh should be used unless there is 'some reasonable reason' not to". Hartman liked the "some reasonable reason" wording and plans to try to work that in.

There was also clear consensus on not using the NMU process to convert packages. The area where consensus is more murky is around whether not using dh should be considered a bug that can be filed against a package. It is clearly not a release-critical (RC) bug, he said, nor would it be a bug if one of the exceptions (or "some reasonable reason") applies. Hartman's best guess is that there may be consensus that not using dh would be considered a normal bug, but that mass-filing bugs is not the way forward either.

While there has been a fair amount of discussion in response, it mostly veered away from either of the two types of responses that Hartman was seeking. That probably indicates that his summary is generally accurate and that participants in the debian-devel mailing list are sanguine about the consensus reached. Kitterman disputed the idea that not using dh was a bug of any sort, but thought that it could be turned into one by changing Debian policy. There is still more than a week to run on the consensus call, but if things stand as they are, Hartman plans to "talk to various people who are impacted about next steps", which presumably includes the policy editors.

To an outside observer, but one who has looked in on Debian discussions a few times over the years, the process has gone much more smoothly than it often does. Jackson called it "an awesome way to conduct this discussion/decisionmaking/whatever". Perhaps this particular set of topics is not as controversial and heat inducing as some, but patiently working through the issues and trying to find common ground where it exists does seem like an improvement. It will be interesting to see where it all goes from here.