Patrick's playground

So here's one of those small things that go beyond the limits of my patience, bit might be very easy for others:

Bug 267267, the virtualbox stabling, is on hold for now. The trigger is a pre-stripped .so. VBox uses kbuild, which has a severe lack of documentation.
I haven't been able to find a way to sanely disable that behaviour. If you have the skills - show the world how awesome you are by writing a patch for any VBox version between 2.1.4 and 2.2.4, attach it to the bug and enjoy that smug feeling of superiority.

If you don't have the skills, don't worry. You'll get there one day if you keep on working on it. Find a simpler bug and try to fix it. Try to fix one bug every day. After 5 years you'll have fixed over a thousand bugs (hey, some days off in between are acceptable) and learned a lot!

In the last weeks I've picked up a few packages mostly because someone pointed me at them and there were lots of open bugs. I've "inherited" virtualbox like that (although Jokey and X-Drum are still doing an awesome job whenever they have the time), I've mostly taken over xen for now (even though I don't use it at the moment and test it in *ahem* VirtualBox).
Samba is another one of those packages that many people use, but few devs maintain. But it's a lot harder to test, so I mostly leave it alone for now.

So I'm beginning to wonder - what packages are "orphaned", where did users provide and test patches and no dev is around?
How can we improve our response time to users so that they are happy and keep helping us, and how do we notice that a package doesn't get the love it deserves?
My current mechanism for that is quite crude and biased - if I notice enough people complaining I have a look, and if the build system doesn't make me want to get drunk I start playing around with it until a few bugs are closed.
Maybe we need a "Package Fix" team of users and devs?
The users could cooperate on collecting bugs for a "topic" like samba and test them, and the devs can give assistance and then commit the fixes. It would most likely be quite fun to cooperate like this, might be a theme for a bugday - but who has the time for that?

If you have an open bug with a fix that hasn't been committed feel free to Mail Me or drop by in #gentoo-bugs on irc.freenode.net. I can't promise much, my time is limited and sometimes things just don't work out as expected, but still I try. And maybe it gets a few old bugs killed - that would be worth it :)

Following everyone else writing a manifesto for the current Council elections I've finally found the time to write one too. You can find it here

I hope that motivates more people to vote. As long as you vote it's good ... come on, it takes all of 5 minutes to vote (and if you really don't care leave the candidates in the random order given by the votify script).

Just to repeat myself, http://dev.gentoo.org/~patrick/Manifesto.html

Here's something that has been bugging me quite a bit lately:
Many projects are unable to simply provide a sane stable download URL for their releases.

In theory it's quite easy. For every release you make a tarball (that can be done easily by your version control system) and put it in a directory (say, /download) on your webserver.

When I want to download a release I just look at the download link on the homepage and either get a direct link to the latest or a directory view of all available items.
Sounds easy, eh?

The problem with that naive approach is that somehow people do not understand HTTP anymore. Instead of providing a simple list it's a web 2.0 bouncy rotating dynamic swirly thing. Which provides me with unclickable links (thank you GitHub!).
Or it's a service like sourceforge providing mirroring. So the "simple" URI gets redirected to the "best" server - which means that a request for the canonical URI gets redirected (HTTP 302), then gets redirected to the mirror (HTTP 301), then gets an OK (HTTP 200) ... for an error page. Which means that if there's any change in the mirror layout or file availability there is NO automated way to detect it because the return code is WRONG.

Which means that I have to manually check every file that gets automatically downloaded to see if it is an HTML error page.
Then there's funny things like Berlios adding a random byte at the end of every archive so that checksums fail (but they stopped that quite a while ago)

And then of course there is the upstream that refuses to provide tarballs because you can do them yourself. Which means that lots of time gets wasted trying to package things in a sane way. (Added bonus: their svn checkout has all libraries included so that you have to find a way of ripping them out. Yey!)

Is it really so hard to configure your webserver to return correct HTTP status codes and just provide simple downloadable archives with a stable URL ? Or is that a black art that takes years to learn?

Y'know, whatever I said about GLEP55 in my last blarghpost isn't relevant anymore.
The current version (1.3) does not mention the things I criticized in the old version (1.2). Maybe they'll be there again in the version up for council discussion next time.
Who knows :)
Because of that I feel the need to point out that any discussion of GLEPs is problematic if one doesn't add the version so we all know what is being talked about (but that doesn't appear to be what people want, next you'll ask for Facts and Logic! Bah!)

Here's a brilliant little thing I found by accident:

Specification
Ebuild filename extension syntax: ebuild[-], where [] denotes an optional part, and  is the EAPI of the ebuild.
Let's call the EAPI included in the ebuild filename the pre-source EAPI, 
and the EAPI set inside the ebuild the post-source EAPI. Given these two, the final EAPI used by 
the ebuild can be established by following these steps:

If the pre-source EAPI is not set it defaults to 0.
If the pre-source EAPI is not recognised it is returned immediately.
If the post-source EAPI is not set, it defaults to the pre-source EAPI.

post-source EAPI is returned.

That is from GLEP55

So for anyone who hasn't noticed, this is a backdoor that invalidates the assumption that the filename represents the actual EAPI. In other words, you'll have to parse the ebuild anyway to find the actual EAPI, the filename is just a hint - which means that you don't gain anything over the current situation.
Now if you were to put the EAPI into a string which is near the top of the file and has to contain the EAPI in unambiguous terms (no shell expansion, variable substitution or other trickeries) you would actually have less work to extract the EAPI (because you do not have to source the ebuild, as a few people have pointed out) and you can keep one file extension so that tools don't have to guess if that whargharbl over there is an ebuild or not.

Oh, I forgot. Current behaviour and QA policies suggest to put the EAPI near the top in an unambiguous string anyway!
Le Sigh!

Because I'm unable to just leave a comment on since I don't have an account on any of the third-party services that are needed to leave a comment. Which is somehow very funny, but also extremely braindamaged.

Now the idea of doing the benchmarks himself wasn't that bad, but peper is using different configuration files for portage and paludis. So the comparison is quite ... uhm ... difficult.

What I do wonder now is why paludis would be slower when reading one set of config files. I don't want to commit to one package manager, so I can't just use an incompatible config, and I'm not going to work with three different sets of config files.
Also I do wonder how his portage timings are off by a factor of five or so from mine ... too many variables and too much unexpected behaviour happening there to get any useful data out of it. It looks like you must not use portage and paludis on the same system if you want any reasonable things happening.
Next benchmarks I'll try to get done when libportage has enough features to be used ...

So today I was a bit unmotivated again and though "Hmm, how do I waste an hour or two of time?"
And as usual some silly benchmarks happened. After all I need to know what my quadcore can do, right?

Since I had a spare chroot lying around I just took that and did the usual incantations:

# time pmerge -uDp sys-apps/portage
 * Resolving...
[ebuild   R   ] sys-apps/portage-2.1.6.7

real    0m8.815s
user    0m5.992s
sys     0m2.264s

# time emerge -uDp sys-apps/portage

Calculating dependencies... done!
[ebuild   R   ] sys-apps/portage-2.1.6.7 

real    0m28.627s
user    0m15.801s
sys     0m9.689s

# time paludis -ip portage:
( two million lines of random output scrolling by )
real    0m58.921s
user    0m36.186s
sys     0m19.345s

Hmm, that is ... unexpected. Let's try something else:

# time paludis -ip kde
real    2m28.890s
user    1m42.810s
sys     0m38.902s

Ouch. Uhm.

# time emerge -up kde
real    0m2.937s
user    0m2.276s
sys     0m0.068s

Oh, let's add -N -D to the options:

# time emerge -uNDp kde
real    0m2.927s
user    0m2.764s
sys     0m0.120s

And pauldis seems to check the distfiles too, so --fetchonly:

# time emerge -up --fetchonly kde
real    0m13.603s
user    0m10.305s
sys     0m0.808s

Just for reference:

# time pmerge -up kde
real    0m1.270s
user    0m1.180s
sys     0m0.080s

# time pmerge -uDp --fetchonly kde
real    0m6.460s
user    0m5.996s
sys     0m0.188s

So ... uhm ... wow.
How does one manage to be slower than portage by a factor of 10 out of the box?

After a nice round of trolling on IRC in #gentoo-council the exherbo brigade decided to leave:

kloeri (n=kloeri@exherbo/developer/kloeri) has left #gentoo-council ("this is too fucking stupid for me")
spb (i=stephen@freenode/developer/exherbo.spb) has left #gentoo-council ("ditto")
ahf (i=ahf@irssi/staff/ahf) has left #gentoo-council ("bonsaikitten is an idiot..")

I bet they are back next week, but in the meantime they get to feel really good about showing us their resolve.
And just to make a point (again) Kloeri decided to bloooooog.

Now the funny bit ...

He retired from Gentoo in 2007. Quote: "I think it's finally time for me and Gentoo to go our seperate ways."

Then he might or might not have quit posting to the gentoo-dev@ mailinglist 3 months later.

Decided to quit using Gentoo another two and a half months later ...

... just to get banned from the gentoo forums half a year later

Then there's the announcement of the gentoo fork that isn't a fork

And finally, close to two years after deciding to leave the announcement to really honestly seriously leaving all gentoo IRC channels

It is really amusing to see people taking years to really really honestly leave that retarded stupid distro that is so badly mismanaged.
Or maybe it is actually quite awesome? How else do you explain people publically leaving, just to return quietly after some time, always sticking around and still claiming that it's all bad?
Ah well. Maybe one day they'll actually do what they say and just leave us be so we can do what we want without all the interference. It can only take a few more years ...

Depending on your timezone and relative inertial frame this months' stealth bugday is more or less over. Due to various reasons noone seems to have thought of sending a notification email to the relevant mailinglists and such, but still some users and quite a few devs managed to be present in #gentoo-bugs to coordinate the assault on the open bugs.
Through spontaneous reorganization the EAPI2 battalion, lead by Betelgeuse, decided to exterminate as many built_with_use checks, replacing them with much more shiny use-deps. That means for users that instead of failing randomly after 3h of compiling with a message like "Rebuild package bar with useflag foo" portage will warn during dependency calculation and thus make it a lot easier to handle.
The process is not done yet, but the decimation has gone quite well so far - decimation might even be a precise description as not all categories are done yet.
Some overly motivated users started looking at ooooold bugs - the kind you filed back in '96 or so and which everyone seems to have forgotten. Closing those might look pointless, but it reduces the amount of open bugs and thus makes searching a lot easier. I spent quite a bit of time breaking things, but that is quite normal when you're finally unmasking packages that have been masked since May 2007 or so. And somehow there have been about as many new bugs as there were closed bugs, so at times it feels really futile.
In the last days I did notice a higher number of package bump request bugs than usual, so maybe some people have read my blog (putting the number of readers near 12) and are doing what I suggested. If any of you read this: Please get up from your chair, turn around three times clockwise and sit down again. Thanks.
Even if unannounced I think this bugday was quite successful. And maybe we'll all have time next month to try again, this time with mailing list notification and all those nice things. Or maybe we just try to make every day a bugday and try to get as much as possible fixed?
In the end you, the users, benefit from our efforts, so maybe you want to get involved and invest some of your time to keep gentoo the bestest distributionism in da hood. Right?