Wed Nov 14 04:14:44 CET 2012

An informal comparison

A few people asked me to write this down so that they can reference it - so here it is.
A completely unscientific comparison between Linux flavours and how they behave:

CentOS 5 (because upgrading is impossible):
             total       used       free     shared    buffers     cached
Mem:          3942       3916         25          0        346       2039
-/+ buffers/cache:       1530       2411

And on the same hardware, doing the same jobs, a Gentoo:
             total       used       free     shared    buffers     cached
Mem:          3947       3781        166          0        219       2980
-/+ buffers/cache:        582       3365
So we use roughly 1/3rd the memory to get the same things done (fileserver), and an informal performance analysis gives us roughly double the IO throughput.
On the same hardware!
(The IO difference could be attributed to the ext3 -> ext4 upgrade and the kernel 2.6.18 -> 3.2.1 upgrade)

Another random data point: A really clumsy mediawiki (php+mysql) setup.
Since php is singlethreaded the performance is pretty much CPU-bound; and as we have a small enough dataset it all fits into RAM.
So we have two processes (mysql+php) that are serially doing things.

Original CentOS install: ~900 qps peak in mysql, ~60 seconds walltime to render a pathological page
Default-y Gentoo: ~1200 qps peak, ~45-50 seconds walltime to render the same page
Gentoo with -march=native in CFLAGS: ~1800qps peak, ~30 seconds render time (this one was unexpected for me!)

And a "move data around" comparison: 63GB in 3.5h vs. 240GB in 4.5h - or roughly 4x the throughput

So, to summarize: For the same workload on the same hardware we're seeing substantial improvements between a few percent and roughly four times the throughput, for IO-bound as well as for CPU-bound tasks. The memory use goes down for most workloads while still getting the exact same results, only a lot faster.

Oh yeah, and you can upgrade without a reinstall.

Posted by Patrick | Permalink