Lately, I am spending quite some time verifying Open-Cannel SSD hardware prototypes in the context of LightNVM. A great deal of these tests is about benchmarking performance (iops) and latency, for which I use fio. Fio is a great tool to test specific I/O patterns. However, when wanting to benchmark a range of I/O configurations in different setups, it is very convenient to automatize the process. In this post, I present a simple python script that allows to automatize fio tests as a function of block size, number of jobs, I/O depth, etc. and stores the test results in csv format. Also, in order to account for variances, the script supports replicating each fio test a configurable number of times, being the average of these the one being stored.
As mentioned, the script is very simple, which makes it easy to adapt. The base script allows to cover a wide range of if configurations, eliminates the environment variances by executing each configuration several times, and stores the average result for each configuration in csv format. The different options supported by fio can be found here (which is an html version of the fio manual: $ man fio). A couple of options worth emphasizing are:
- –minimal: fio results will be printed in a semicolon-delimited format. This makes it simpler to script the results.
- –time_based –runtime=X: In order to also minimize workload variances (e.g., caching, I/O size) we make sure that each test executes a predetermined number of seconds represented by X.
- –group_reporting: When executing several jobs (threads) fio will report information for each one separately. This option makes the output to be grouped, making it easier to compare different I/O patters in different setups.
This is the current version of the script. The updated version of the python script can be found in my GitHub (here). In the future, I intend to complete the script with parameters and new fio options. I also plan to add gnuplot support in order to automatically generate graphics from the csv file.
import os import time import subprocess device = 'nvme0n1' fio_size="1G" # size in fio fio_runtime="5" # runtime in fio for time_based tests # fio --minimal hardcoded positions fio_iops_pos=7 fio_slat_pos_start=9 fio_clat_pos_start=13 fio_lat_pos_start=37 kernel_version = os.uname() columns="iotype;bs;njobs;iodepth;iops;slatmin;slatmax;slatavg;clatmin;clatmax;clatavg;latmin;latmax;latavg" # Eliminate noise by running each test n times and calculating average. n_iterations=3 f = open(kernel_version + time.strftime("%H%M%S") + "-fio.csv", "w+") f.write(columns+"\n") for run in ('write', 'randwrite', 'read', 'randread'): for blocksize in ('512', '1k', '4k', '512k'): for numjobs in (1, 32, 64): for iodepth in (1, 8, 32, 64, 128): fio_type_offset = 0 iops = 0.0 slat = [0.0 for i in range(3)] clat = [0.0 for i in range(3)] lat = [0.0 for i in range(3)] result = "" + str(run) + ";" + str(blocksize) + ";" + str(numjobs) + ";" + str(iodepth) + ";" command = "sudo fio --minimal -name=temp-fio --bs="+str(blocksize)+" --ioengine=libaio --iodepth="+str(iodepth)+" --size="+fio_size+" --direct=1 --rw="+str(run)+" --filename=/dev/"+str(device)+" --numjobs="+str(numjobs)+" --time_based --runtime="+fio_runtime+" --group_reporting" print (command) for i in range (0, n_iterations): os.system("sleep 2") #Give time to finish inflight IOs output = subprocess.check_output(command, shell=True) if "write" in run: fio_type_offset=41 # fio is called with --group_reporting. This means that all # statistics are group for different jobs. # iops iops = iops + float(output.split(";")[fio_type_offset + fio_iops_pos]) # slat for j in range (0, 3): slat[j] = slat[j] + float(output.split(";")[fio_type_offset+fio_slat_pos_start+j]) # clat for j in range (0, 3): clat[j] = clat[j] + float(output.split(";")[fio_type_offset+fio_clat_pos_start+j]) # lat for j in range (0, 3): lat[j] = lat[j] + float(output.split(";")[fio_type_offset+fio_lat_pos_start+j]) # iops result = result+str(iops / n_iterations) # slat for i in range (0, 3): result = result+";"+str(slat[i] / n_iterations) # clat for i in range (0, 3): result = result+";"+str(clat[i] / n_iterations) # lat for i in range (0, 3): result = result+";"+str(lat[i] / n_iterations) print (result) f.write(result+"\n") f.flush() f.closed
How did you know where to hard code the starting positions at? Also, why is the offset sometimes 0 and sometimes 41?
At the time, these where the offsets when using –minimal. Today, you probably want to use an json output, which is easier to parse.