ext3 versus ext4 — simple benchmark and some issues

Tests were performed on 50GB EBS with provisioned 300 IOPS that has following results in hdparm test:

[code language=”shell”]
root@ip-10–251–3–125:/home# hdparm -tT /dev/xvdc1
/dev/xvdc1:
Timing cached reads: 6312 MB in 1.99 seconds = 3165.79 MB/sec
Timing buffered disk reads: 36 MB in 3.02 seconds = 11.91 MB/sec
[/code]

So first EXT3:

[code language=”shell”]
root@ip-10–251–3–125:/home# time mkfs.ext3 /dev/xvdc1
[…]
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

real 4m50.608s
user 0m0.048s
sys 0m2.328s
[/code]

EXT4:

[code language=”shell”]
root@ip-10–251–3–125:/home# time mkfs.ext4 /dev/xvdc1
mke2fs 1.42 (29-Nov-2011)
[…]
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

real 0m44.398s
user 0m0.184s
sys 0m0.644s
[/code]

So we’ve got 44 seconds on ext4 versus almost 5 minutes on ext3 probably due to delayed allocation. Simply speaking, ext4 is leaving vast of your filesystem blank unless you decide to put some data on your hard drive. When this happens, ext4 will gather the data and perform delayed commit onto the device previously creating formatted space on it.

I have dealed with Debian 6 machine that generated something like 10% of iowait on an very-low-traffic mysqld service [1]. The problem was caused by bad support of ext4 in 2.6.32 kernel. Installing 3.2 from backports resolved the issue.

[1] This and also this post is adequately describing the problem.
[2] Proper screenshot of the issue from the monitoring system. You can see the funny trapezoidal spikes that were caused by ext4 making more and more space for itself.