Gentoo's growing pains

[Posted May 3, 2006 by ris]

A post to the Gentoo-devel mailing entitled Gentoo: State of the Union and the discussion that followed show that Gentoo is having some growing pains. It's not the first or the only sign, but the thread covers most of the major signs.

Gentoo now has more than 300 developers and over ten thousand packages in portage, a size that rivals Debian, and it got there in a fairly short period of time. Some growing pains are a natural consequence of that growth.

Topics in this discussion include the ease (or lack thereof) of becoming a Gentoo developer, the usefulness of GLEPs (Gentoo Linux Enhancement Proposals), separating a development tree from a stable tree, voting, source management control systems and more.

How easy should it be to become a developer? Anyone should be able to jump in and contribute, but that doesn't mean they should be granted commit access right away. Granting commit access too easily creates problems, usually due to the errors of inexperienced people. If the process is easy enough, it's only a matter of time before someone with malicious intent starts mucking with the tree. Currently Gentoo requires prospective developers to take a quiz. There is generally some mentoring that to help the person get ready for the quiz. Once a person passes the quiz they should know enough about how Gentoo works to avoid commit errors. The malicious are not likely to work that hard and the mentor has a good chance of weeding them out before they get that far in any case.

The process does get bogged down when there are not enough mentors. Not every developer makes a good mentor. Even those who are good mentors may have personality conflicts with some people. This problem is not unique to Gentoo. Overall, it seems that becoming a Gentoo developer is easy enough to attract a steady stream of new people, but commit access is restrictive enough to prevent major problems.

GLEPs may be proposed by users or developers. They get written up the GLEP editors and posted to the development list for discussion. During the discussion the GLEP is revised. Some die during the recursive iterations, some go on to a vote. If the GLEP only affects a single team it will be voted on by that team. GLEPs with broader implications are voted on by the Gentoo Council. Even if the GLEP passes, it may not be implemented. This is not ideal, but at least the trail of dead GLEPs provide insight to bad ideas and keep them from being proposed over and over and over again.

Gentoo still has much to work out. The project has the advantage of seeing what works (and what doesn't) in the Debian project. They have the opportunity of making all new mistakes as the project deals with its growth and popularity. From an editorial standpoint it can be fun to watch.

to post comments

Gentoo's growing pains

Posted May 4, 2006 16:25 UTC (Thu) by lovelace (guest, #278) [Link] (2 responses)

They get written up the GLEP editors and posted to the development list for discussion

Should that instead be

"They get written up by the GLEP editors and posted to the development list for discussion"

Gentoo's growing pains

Posted May 4, 2006 16:31 UTC (Thu) by g2boojum (subscriber, #152) [Link] (1 responses)

Not quite (thank goodness!). It should read "They get written up, submitted to the GLEP editors for a sanity check, and posted to the development list for discussion".

Gentoo's growing pains

Posted May 4, 2006 16:38 UTC (Thu) by lovelace (guest, #278) [Link]

Ah! That makes a lot more sense. Thanks for the clarification.

Gentoo's growing pains

Posted May 4, 2006 16:44 UTC (Thu) by g2boojum (subscriber, #152) [Link]

Thank you for the kind words!

"From an editorial standpoint it can be fun to watch."

I'm happy to say that it is still fun to develop, too.

Gentoo's growing pains

Posted May 4, 2006 22:22 UTC (Thu) by kamil (guest, #3802) [Link] (9 responses)

I wonder if a problem of slow migration of various packages to new versions is another indication of "growing pains"...

As an example, of the software that I really depend on, teTeX on Gentoo is still version 2.0.2 (mainline 3.0 was released 15 months ago), and Firefox is 1.0.8 (mainline 1.5 was released 6 months ago).

Sure, I can always unmask some packages to access the unstable versions, and I do, but why does it take so long to get major Linux components in shape? I expect a source-based distribution to be more bleeding-edge than that...

Gentoo's growing pains

Posted May 4, 2006 23:05 UTC (Thu) by bk (guest, #25617) [Link]

Part of that is an attempt at better QA. Rather than simply unmasking packages en masse after a certain period of time, they wait until there is real evidence that the items are stable and without major outstanding bugs. That way when you choose to run Gentoo stable, it really is stable.

Gentoo's growing pains

Posted May 5, 2006 4:52 UTC (Fri) by tetromino (guest, #33846) [Link] (1 responses)

Wrong terminology.

You only need to unmask packages that are masked (listed in /usr/portage/profiles/package.mask); and packages get masked because they are beta, or buggy, or incompatible with the rest of the OS, or have security issues. For instance, gcc-4.2.0_alpha20060429 is masked, for obvious reasons.

The word you are looking for is keywording. You have decided to only install packages that are keyworded "x86". That gives you Gentoo's analogue of Debian Stable. It's stable, but it ain't current. If you want more current packages, set your ACCEPT_KEYWORDS to "x86 ~x86", somewhat analagous to Debian Testing/Unstable, which would have given you tetex 3 and firefox 1.5 as soon as they were released. Of course, you would have had to recompile firefox 1.5 ten times as the Gentoo devs gradually figured out how to handle the package properly -- but that's what ~x86 is for.

Gentoo's growing pains

Posted May 10, 2006 11:13 UTC (Wed) by tres (guest, #352) [Link]

set your ACCEPT_KEYWORDS to "x86 ~x86"

That works pretty well on x86 but not so well on some of the other archs. AMD64 is getting very close to being able to use the "~amd64" keyword globally but there may still be some corner cases. I fear that some of the other archs are a bit farther behind but your milage may vary.

Gentoo's growing pains

Posted May 5, 2006 17:08 UTC (Fri) by yodermk (subscriber, #3803) [Link] (3 responses)

And KDE is still at 3.4! 3.5 has been out for, what, 6 months now?

I had decided to keep KDE stable (instead of ~x86), but I'm starting to get frustrated. It's not as bad as Debian, but still ...

Gentoo's growing pains

Posted May 6, 2006 4:17 UTC (Sat) by set (guest, #4788) [Link] (2 responses)

ACCEPT_KEYWORDS="~x86" emerge -pv kde

viola, you would get KDE 3.5.2

Its not an all or nothing proposition. If you want something
thats outside of 'stable', just add it. See also /etc/portage/package.keywords
(although for something as komplex as KDE that might
be a pain.) I personally have it setup to get 'the latest' on
a certain set of applications I want bleeding.

For somethings I maintain a local portage overlay; 90% of the time
you can just copy foo-app-1.2.3.ebuild to foo-app-1.2.4.ebuild
and it will just work. (after rebuilding the digest)

There are also a lot of specialty externally maintained portage
overlays maintained by people who arent official devs.

Gentoo's growing pains

Posted May 6, 2006 17:13 UTC (Sat) by yodermk (subscriber, #3803) [Link]

Yeah I'm well aware that can be done, and it's one of the things I really like about Gentoo. But as I said in grandparent, I had just decided to keep KDE stable for now.

I never expected it to be this long. KDE 3.4 was in stable at the .1, only 3-4 months after 3.4.0 IIRC.

Gentoo's growing pains

Posted May 10, 2006 11:13 UTC (Wed) by tres (guest, #352) [Link]

ACCEPT_KEYWORDS="~x86" emerge -pv kde

Please don't do this. It is highly discouraged to use ACCEPT_KEYWORDS (or USE) on the command line as portage will not remember anything about those settings. The proper file, as mentioned, is /etc/portage/package.keywords (and /etc/portage/package.use) and that will enable portage to remember the settings across calls.

Gentoo's growing pains

Posted May 7, 2006 5:09 UTC (Sun) by dirtyepic (guest, #30178) [Link] (1 responses)

Gentoo Release Engineering Lead Chris Gianelloni posted a response to the question of 'why does it take so long for a package to go stable' on the gentoo-dev ml today. I hope he won't mind be quoting it.

> > As a user I have to add my opinion here. I have been using Gentoo for some
> > years now and it was always fairly up to date. Currently KDE is really
> > behind on the current situation upstream.
> > And then I wonder why. What makes us think we can not trust the KDE devs?
> > Does compiling KDE introduce so many bugs? I mean, let's be serious, all
> > other distributions have a stable 3.5.x now. Don't they experience all
> > those horrible bugs?

Compiling KDE doesn't introduce bugs. Compiling KDE with any
combination of USE/CFLAGS/CXXFLAGS/GCC/Glibc/etc does. Remember that
we're a from-source distribution. Guys like Debian or Red Hat just have
to compile it *once* then they make a package of it, with exactly *one*
set of options (USE), C(XX)FLAGS, gcc, glibc, etc. making their job
infinitely easier.

> > Seriously, this is really becoming an issue. As I pointed out in a bug I
> > filed for a stable KDE (for which I apologize, I should have looked here
> > first), some people are leaving Gentoo because of this slow upgrade
> > process.

Honestly, if they're leaving over something so minor, they're free to
go. We're not a commercial distribution. We don't sell Gentoo. We're
not concerned with market share.

> > The classical answer from devs is "it's ready when it's ready". From a user
> > point of view this is very, very vague. Please give users a more clear
> > explanation, this creates great frustration when looking at other
> > distributions. Because it's stable there.

As I stated above, they have a *much* lower barrier of entry for making
something stable than we do. We've tried making this explanation over
and over again. The problem is that every single version of $package,
people don't look at the last explanation and ask again... and again...
and again... and again. It gets very old to answer the same question
over and over again. The simple answer is really "when we don't have
major showstopper bugs anymore". Again, remember that we have to
support countless combinations from our users. Other distributions need
to support only one. They can use forms of tricks to get it to compile
that *one* time, including adding patches and other things that might
not be suitable for a from-source distribution.

> > These are my 2 cents as a user. One that loves Gentoo. One that loves KDE..
> > One that's frustrated by the current situation. I am a CS so I know how
> > hard programming can be, don't get me wrong there. I do appreciate what you
> > guys do. But I can't understand why you do it this way right now.

Quite simply, we don't want to give you crap.

If we followed others blindly, as so many users suggest, then we would
have stabilized KDE 3.5 ages ago, and every single one of you KDE users
would be complaining about how our QA sucks because KDE doesn't compile
or breaks badly in so many places. We would hear about how Gentoo sucks
where they can't even test such a major application as KDE properly. We
would have users leaving in droves. Now, we can't have both fast
stabilization *and* actual stability, so we err on the side of caution.
We don't like hearing complaints any more than anyone else, but we'd
rather hear a few "why isn't KDE stable yet" questions than *everyone*
saying we suck because KDE is broken.

I hope that sums it up for you.

By the way, this isn't just for KDE. This is how we do *every* package.

Gentoo's growing pains

Posted May 8, 2006 14:10 UTC (Mon) by kamil (guest, #3802) [Link]

Thanks for cross-posting this, it was very informational.

Sadly, the outcome of that posting might be: those who want to run current versions of major software packages with some minimal guarantees of stability, should consider switching to other distributions.

I'm not saying it's the fault of the Gentoo project; rather, a possible indication that the Gentoo philosophy (build everything yourself) is just not the right way in a long run...

ACCEPT_KEYWORDS, posted by others, is a possible workaround, but I'm reluctant to even try it: it seems too blunt a tool. How is a user not intimately involved in Gentoo development supposed to distinguish packages that just don't build with some esoteric compiler options (possibly on another platform) from those that are fundamentally incompatible with the rest of the system (or plain broken) and will render the system unusable upon emerging?