Mon Jul 8 06:59:41 CEST 2013

Explaining AutoRepoman

Goal of this exercise: Have a cronjob report on IRC when a QA violation happens

Step one: Create repoman output for each category.
Difficulty: Repoman takes ~135minutes to run
Difficulty: default output included masked profiles, so run with -I

parallelize () {
        while [ $# -gt 0 ] ; do
                jobcnt=(`jobs -p`)
                if [ ${#jobcnt[@]} -lt $maxjobs ] ; then
                        do-something $1 &
do-something() {
        cd /usr/portage/$1
        date > repoman-checks/$1.txt
        repoman -I full > repoman-checks/$1.txt
        repoman full > repoman-checks/full/$1.txt
parallelize `cat /usr/portage/profiles/categories`
Tadaah, we spend 4 CPU-hours to generate per-category files (with perfect parallelization that's ~30minutes wall time).
You can see them here

Step two: Find all new badness
Difficulty: Don't just spam everything
Solution: grep for "interesting" patterns - right now that's (R)DEPEND and metadata.xml breakage.
mv repoman-new.txt repoman-old.txt
grep -h -e DEPEND: repoman-checks/* | cut -d ":" -f 1,3- | cut -d "/" -f 1,3-| sed -e 's/~.*)//' | sort -u | grep -v consider > repoman-new.txt
grep -h -e metadata.xml repoman-checks/* >> repoman-new.txt
comm -23 repoman-new.txt repoman-old.txt > repoman-diff.txt
while read -r; do  
        echo $REPLY
        ssh commitbot@somewhere "{\"to\": [\"irc://\"], \"privmsg\": \" "AutoRepomanWarning: $REPLY" \"}" &
done < repoman-diff.txt
Et voila, we only send each new line once to #gentoo-bugs so that maybe people notice these issues and fix them. It uses irker, like the #gentoo-commits notifications.

And if someone wants a new check added - just add a new grep there and wait for the cronjob to run.

Posted by Patrick | Permalink