Testing IO performance in Linux [closed]

IO and filesystem benchmark is a complex topic. No single benchmarking tool is good in all situations. Here is a small overview about different benchmarking tools:

Block Storage:

• IOMeter - Highly customizable and allows to coordinate multiple clients. Needs a Windows PC for the coordination application. Developed by Intel. On Linux, take maximum rates of older (at least 2006.07.27 and earlier) with a pinch of salt because the submission method was not optimal.

File System (synthetic):

• FFSB - Flexible Filesystem Benchmark. Very neat benchmarking for Linux. Good customization of workload. NFS benchmarking (net-ffsb) a bit unsound.
• Filebench - Extremely powerful, but originally developed for Solaris. Linux support isn't good.
• sysbench - Mainly a DB benchmarking tool, but also basic filesystem benchmarking tool.
• bonnie - Seems to be obsolete.
• bonnie++ - C++ port of bonnie. Easy, but seems not to be very customizable.

File System (workload):

• Postmark - Simulates the IO behavior of a mail server. Too small to stress good IO systems.

Stony Brook University and IBM Watson Labs have published a highly recommended journal paper in the "Transaction of Storage" about file system benchmarking, in which they present different benchmarks and their strong and weak points: A nine year study of file system and storage benchmarking. The article clearly points out that the results of most benchmarks at least questionable.

A note: Is the question programming related? Maybe not, but maybe it is. I spend a lot of time benchmarking the IO performance of the systems I develop. At least for me, questions about how to benchmarking these things is highly programming related. Please: Do not close all questions that are not development/programming related from your point of view. The point of view of other developers might be different.

tool: fio

link: http://freshmeat.net/projects/fio/

test physical disk IO:

    ./fio examples/disk-zone-profile 

set parameter: sequential r/w: rw=read or rw=write random r/w: rw=randread or rw=randwrite

if you need a quick way without hassle of installing anything . This is the method I use for write speed test:

dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync

And the output is something like this

root@rackserver:/# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 4.86922 s, 221 MB/s

Also : delete the test file after this to recover the extra space used

Some explanation :

bs = block size
count = the no of blocks to be written

Adjust these parameters to change the size of the file written as per your server specs and the amount of time you want to spend writing.

the read speed as suggested already by gtsouk, can be checked by using /dev/null as output.

dd if=/dev/sda of=/dev/null

Let this run for a few minutes and stop it with ctrl+C. It will print the read transfer speed of your drive/controller. This is the maximum read speed you can get out of your drive.

you need to specify what you're testing for, otherwise benchmarks will only mislead. There are different aspects of IO performance that you need to chose to optimize for, and different parameters to play with.

Your system parameters:

1. storage device: HDD, SSD (which?), Raid (which?)
2. filesystem, block size, journal mode
3. file cache, dirty thresholds, amount of memory
4. IO scheduler, its tunables
5. number of CPUs
6. kernel version

Your test parameters:

1. read or write performance?
2. sequential or random?
3. 1 thread or multiple?
4. size of requests
5. optimize for throughput or request delay?

There is an excellent program to test block storage IO on Unix called IORATE. You can get a copy at iorate.org.

It can generate complex mixed IO, including re-use (hits) and hot zones for tiered storage testing.

sysbench

See http://www.howtoforge.com/how-to-benchmark-your-system-cpu-file-io-mysql-with-sysbench

Example

sysbench --test=fileio --file-total-size=150G prepare

sysbench --test=fileio --file-total-size=150G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run

It can also test cpu, memory, threads, and database server performance,

It's awesome.

Or testing software written in java: http://www.dacapobench.org/

Take a look at IOzone: http://www.iozone.org/

If you would like to read a whitepaper illustrating real-world usage on an HPC cluster, please see this pdf, page 36: http://i.dell.com/sites/content/business/solutions/hpcc/en/Documents/Dell-NSS-NFS-Storage-solution-final.pdf