Mon Jul 8 06:59:41 CEST 2013
Explaining AutoRepoman
Goal of this exercise: Have a cronjob report on IRC when a QA violation happens
Step one: Create repoman output for each category.
Difficulty: Repoman takes ~135minutes to run
Difficulty: default output included masked profiles, so run with -I
You can see them here
Step two: Find all new badness
Difficulty: Don't just spam everything
Solution: grep for "interesting" patterns - right now that's (R)DEPEND and metadata.xml breakage.
And if someone wants a new check added - just add a new grep there and wait for the cronjob to run.
Step one: Create repoman output for each category.
Difficulty: Repoman takes ~135minutes to run
Difficulty: default output included masked profiles, so run with -I
#!/bin/bash maxjobs=8 parallelize () { while [ $# -gt 0 ] ; do jobcnt=(`jobs -p`) if [ ${#jobcnt[@]} -lt $maxjobs ] ; then do-something $1 & shift fi done wait } do-something() { cd /usr/portage/$1 date > repoman-checks/$1.txt repoman -I full > repoman-checks/$1.txt repoman full > repoman-checks/full/$1.txt } parallelize `cat /usr/portage/profiles/categories`Tadaah, we spend 4 CPU-hours to generate per-category files (with perfect parallelization that's ~30minutes wall time).
You can see them here
Step two: Find all new badness
Difficulty: Don't just spam everything
Solution: grep for "interesting" patterns - right now that's (R)DEPEND and metadata.xml breakage.
#!/bin/bash mv repoman-new.txt repoman-old.txt grep -h -e DEPEND: repoman-checks/* | cut -d ":" -f 1,3- | cut -d "/" -f 1,3-| sed -e 's/~.*)//' | sort -u | grep -v consider > repoman-new.txt grep -h -e metadata.xml repoman-checks/* >> repoman-new.txt comm -23 repoman-new.txt repoman-old.txt > repoman-diff.txt while read -r; do echo $REPLY ssh commitbot@somewhere "{\"to\": [\"irc://chat.freenode.net/#gentoo-bugs\"], \"privmsg\": \" "AutoRepomanWarning: $REPLY" \"}" & done < repoman-diff.txtEt voila, we only send each new line once to #gentoo-bugs so that maybe people notice these issues and fix them. It uses irker, like the #gentoo-commits notifications.
And if someone wants a new check added - just add a new grep there and wait for the cronjob to run.