Home > Uncategorized > I/O performance and stripe_cache_size

I/O performance and stripe_cache_size

Results from a basic synthetic benchmark of I/O performance whilst varying the stripe_cache_size tunable under the sysfs tree at /sys/block/md0/md/stripe_cache_size

Tests were performed on a QNAP SS-439 NAS  :

  • Intel Atom N270 (1.6GHz) 82801G
  • 2GB RAM
  • Linux kernel
  • CFQ I/O elevator
  • RAID5  (128kb chunksize) – 4* WD10TPVT 1TB drives (4kb physical sectors, aka. Advanced Format)
  • EXT3 filesystem (noatime)

Whilst reading and writing in blocks of just 512 bytes, there seems to be no discernible benefit in setting a larger stripe cache size, with read performance dropping marginally as the cache size is increased.

The first interesting results appeared when the blocksize was increased to 4096bytes. Read performance drops off sharply as we increase the cache, though write performance gains a small amount.

At a blocksize of 1MB, our previous findings are reinforced. Read performance decreases significantly once past very low cache sizes, though write performance benefits a small amount by using larger values for the stripe_cache_size.

  1. May 14, 2011 at 08:02

    I’ve also ran some similar tests but I’m seeing very different results. My setup is also a bit different (AMD Phenom II X4 840, 4GB, Linux 2.6.38, 4x Deskstar 7K3000 3TB, 512b sectors, 512kb chunk size). My results with a simple ‘dd’ directly on the MD device, 1MB blocks:

    stripe_cache_size = 256:
    Read: 409 MB/s
    Write: 55 MB/s

    stripe_cache_size = 1024:
    Read: 420 MB/s
    Write: 145 MB/s

    stripe_cache_size = 32768:
    Read: 423 MB/s
    Write: 300 MB/s

    I.e. a huge performance boost for write performance when increasing the size of the stripe cache. My guess is that you’re being held back by the performance of the memory subsystem of the Atom platform.

  2. Paul
    May 19, 2011 at 09:43

    You have to remember that increasing the stripe cache size quickly leads to increased memory usage leaving less available for the buffers. The formula to find memory usage is data drives x 4k x stripe_cache_size. For a 16 MB stripe cache, that’s 3 x 4 x 16384 = 192 MB RAM allocated to be used solely for writing only. 32 MB stripe cache is 384 MB and almost 20% of the system memory, so it makes sense that read speed will drop off once the stripe cache reaches a certain point.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: