fio

fio(1) General Commands Manual fio(1)

NAME

   fio - flexible I/O tester

SYNOPSIS

   fio [options] [jobfile]...

DESCRIPTION

   fio  is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.  The typical use of fio is to write a job file matching
   the I/O load one wants to simulate.

OPTIONS

   --debug=type
          Enable verbose tracing type of various fio actions. May be `all' for all types or individual types separated by a comma (e.g. `--debug=file,mem' will enable file and memory de‐
          bugging). `help' will list all available tracing options.

   --parse-only
          Parse options only, don't start any I/O.

   --merge-blktrace-only
          Merge blktraces only, don't start any I/O.

   --output=filename
          Write output to filename.

   --output-format=format
          Set the reporting format to `normal', `terse', `json', or `json+'. Multiple formats can be selected, separate by a comma. `terse' is a CSV based format. `json+' is like `json',
          except it adds a full dump of the latency buckets.

   --bandwidth-log
          Generate aggregate bandwidth logs.

   --minimal
          Print statistics in a terse, semicolon-delimited format.

   --append-terse
          Print statistics in selected mode AND terse, semicolon-delimited format.  Deprecated, use --output-format instead to select multiple formats.

   --terse-version=version
          Set terse version output format (default `3', or `2', `4', `5').

   --version
          Print version information and exit.

   --help Print a summary of the command line options and exit.

   --cpuclock-test
          Perform test and validation of internal CPU clock.

   --crctest=[test]
          Test the speed of the built-in checksumming functions. If no argument is given, all of them are tested. Alternatively, a comma separated list can be passed, in which  case  the
          given ones are tested.

   --cmdhelp=command
          Print help information for command. May be `all' for all commands.

   --enghelp=[ioengine[,command]]
          List all commands defined by ioengine, or print help for command defined by ioengine. If no ioengine is given, list all available ioengines.

   --showcmd
          Convert given jobfiles to a set of command-line options.

   --readonly
          Turn  on  safety read-only checks, preventing writes and trims. The --readonly option is an extra safety guard to prevent users from accidentally starting a write or trim work‐
          load when that is not desired. Fio will only modify the device under test if `rw=write/randwrite/rw/randrw/trim/randtrim/trimwrite' is given. This safety net can be used as  an
          extra precaution.

   --eta=when
          Specifies  when  real-time  ETA estimate should be printed. when may be `always', `never' or `auto'. `auto' is the default, it prints ETA when requested if the output is a TTY.
          `always' disregards the output type, and prints ETA when requested. `never' never prints ETA.

   --eta-interval=time
          By default, fio requests client ETA status roughly every second. With this option, the interval is configurable. Fio imposes a minimum allowed time to avoid flooding  the  con‐
          sole, less than 250 msec is not supported.

   --eta-newline=time
          Force a new line for every time period passed. When the unit is omitted, the value is interpreted in seconds.

   --status-interval=time
          Force  a  full status dump of cumulative (from job start) values at time intervals. This option does *not* provide per-period measurements. So values such as bandwidth are run‐
          ning averages. When the time unit is omitted, time is interpreted in seconds. Note that using this option with `--output-format=json' will yield output that  technically  isn't
          valid json, since the output will be collated sets of valid json. It will need to be split into valid sets of json after the run.

   --section=name
          Only  run  specified  section  name  in job file. Multiple sections can be specified.  The --section option allows one to combine related jobs into one file.  E.g. one job file
          could define light, moderate, and heavy sections. Tell fio to run only the "heavy" section by giving `--section=heavy' command line option. One can also specify the "write" op‐
          erations in one section and "verify" operation in another section. The --section option only applies to job sections. The reserved *global* section is always parsed and used.

   --alloc-size=kb
          Allocate additional internal smalloc pools of size kb in KiB. The --alloc-size option increases shared memory set aside for use by fio.  If running large  jobs  with  randommap
          enabled,  fio  can  run out of memory.  Smalloc is an internal allocator for shared structures from a fixed size memory pool and can grow to 16 pools. The pool size defaults to
          16MiB.  NOTE: While running `.fio_smalloc.*' backing store files are visible in `/tmp'.

   --warnings-fatal
          All fio parser warnings are fatal, causing fio to exit with an error.

   --max-jobs=nr
          Set the maximum number of threads/processes to support to nr.  NOTE: On Linux, it may be necessary to increase the shared-memory limit (`/proc/sys/kernel/shmmax') if  fio  runs
          into errors while creating jobs.

   --server=args
          Start a backend server, with args specifying what to listen to.  See CLIENT/SERVER section.

   --daemonize=pidfile
          Background a fio server, writing the pid to the given pidfile file.

   --client=hostname
          Instead of running the jobs locally, send and run them on the given hostname or set of hostnames. See CLIENT/SERVER section.

   --remote-config=file
          Tell fio server to load this local file.

   --idle-prof=option
          Report CPU idleness. option is one of the following:

                 calibrate
                        Run unit work calibration only and exit.

                 system Show aggregate system idleness and unit work.

                 percpu As system but also show per CPU idleness.

   --inflate-log=log
          Inflate and output compressed log.

   --trigger-file=file
          Execute trigger command when file exists.

   --trigger-timeout=time
          Execute trigger at this time.

   --trigger=command
          Set this command as local trigger.

   --trigger-remote=command
          Set this command as remote trigger.

   --aux-path=path
          Use the directory specified by path for generated state files instead of the current working directory.

JOB FILE FORMAT

   Any  parameters following the options will be assumed to be job files, unless they match a job file parameter. Multiple job files can be listed and each job file will be regarded as a
   separate group. Fio will stonewall execution between each group.

   Fio accepts one or more job files describing what it is supposed to do. The job file format is the classic ini file, where the names enclosed in [] brackets define the job  name.  You
   are  free to use any ASCII name you want, except *global* which has special meaning. Following the job name is a sequence of zero or more parameters, one per line, that define the be‐
   havior of the job. If the first character in a line is a ';' or a '#', the entire line is discarded as a comment.

   A *global* section sets defaults for the jobs described in that file. A job may override a *global* section parameter, and a job file may even have several *global* sections if so de‐
   sired. A job is only affected by a *global* section residing above it.

   The --cmdhelp option also lists all options. If used with an command argument, --cmdhelp will detail the given command.

   See the `examples/' directory for inspiration on how to write job files. Note the copyright and license requirements currently apply to `examples/' files.

   Note that the maximum length of a line in the job file is 8192 bytes.

JOB FILE PARAMETERS

   Some parameters take an option of a given type, such as an integer or a string. Anywhere a numeric value is required, an arithmetic expression may be used, provided it  is  surrounded
   by parentheses. Supported operators are:

          addition (+)

          subtraction (-)

          multiplication (*)

          division (/)

          modulus (%)

          exponentiation (^)

   For time values in expressions, units are microseconds by default. This is different than for time values not in expressions (not enclosed in parentheses).

PARAMETER TYPES

   The following parameter types are used.

   str    String. A sequence of alphanumeric characters.

   time   Integer  with possible time suffix. Without a unit value is interpreted as seconds unless otherwise specified. Accepts a suffix of 'd' for days, 'h' for hours, 'm' for minutes,
          's' for seconds, 'ms' (or 'msec') for milliseconds and 'us' (or 'usec') for microseconds. For example, use 10m for 10 minutes.

   int    Integer. A whole number value, which may contain an integer prefix and an integer suffix.

                 [*integer prefix*] **number** [*integer suffix*]

          The optional *integer prefix* specifies the number's base. The default is decimal. *0x* specifies hexadecimal.

          The optional *integer suffix* specifies the number's units, and includes an optional unit prefix and an optional unit. For quantities of data, the default unit  is  bytes.  For
          quantities of time, the default unit is seconds unless otherwise specified.

          With `kb_base=1000', fio follows international standards for unit prefixes. To specify power-of-10 decimal values defined in the International System of Units (SI):

                 K means kilo (K) or 1000
                 M means mega (M) or 1000**2
                 G means giga (G) or 1000**3
                 T means tera (T) or 1000**4
                 P means peta (P) or 1000**5

          To specify power-of-2 binary values defined in IEC 80000-13:

                 Ki means kibi (Ki) or 1024
                 Mi means mebi (Mi) or 1024**2
                 Gi means gibi (Gi) or 1024**3
                 Ti means tebi (Ti) or 1024**4
                 Pi means pebi (Pi) or 1024**5

          For Zone Block Device Mode:

                 z means Zone
          With `kb_base=1024' (the default), the unit prefixes are opposite from those specified in the SI and IEC 80000-13 standards to provide compatibility with old scripts. For exam‐
          ple, 4k means 4096.

          For quantities of data, an optional unit of 'B' may be included (e.g., 'kB' is the same as 'k').

          The *integer suffix* is not case sensitive (e.g., m/mi mean mebi/mega, not milli). 'b' and 'B' both mean byte, not bit.

          Examples with `kb_base=1000':

                 4 KiB: 4096, 4096b, 4096B, 4k, 4kb, 4kB, 4K, 4KB
                 1 MiB: 1048576, 1m, 1024k
                 1 MB: 1000000, 1mi, 1000ki
                 1 TiB: 1073741824, 1t, 1024m, 1048576k
                 1 TB: 1000000000, 1ti, 1000mi, 1000000ki

          Examples with `kb_base=1024' (default):

                 4 KiB: 4096, 4096b, 4096B, 4k, 4kb, 4kB, 4K, 4KB
                 1 MiB: 1048576, 1m, 1024k
                 1 MB: 1000000, 1mi, 1000ki
                 1 TiB: 1073741824, 1t, 1024m, 1048576k
                 1 TB: 1000000000, 1ti, 1000mi, 1000000ki

          To specify times (units are not case sensitive):

                 D means days
                 H means hours
                 M mean minutes
                 s or sec means seconds (default)
                 ms or msec means milliseconds
                 us or usec means microseconds

          `z' suffix specifies that the value is measured in zones.  Value is recalculated once block device's zone size becomes known.

          If  the  option  accepts  an upper and lower range, use a colon ':' or minus '-' to separate such values. See irange parameter type.  If the lower value specified happens to be
          larger than the upper value the two values are swapped.

   bool   Boolean. Usually parsed as an integer, however only defined for true and false (1 and 0).

   irange Integer range with suffix. Allows value range to be given, such as 1024-4096. A colon may also be used as the separator, e.g. 1k:4k. If the option allows two  sets  of  ranges,
          they can be specified with a ',' or '/' delimiter: 1k-4k/8k-32k. Also see int parameter type.

   float_list
          A list of floating point numbers, separated by a ':' character.

JOB PARAMETERS

   With the above in mind, here follows the complete list of fio job parameters.

Units

   kb_base=int
          Select the interpretation of unit prefixes in input parameters.

                 1000   Inputs comply with IEC 80000-13 and the International System of Units (SI). Use:

                        - power-of-2 values with IEC prefixes (e.g., KiB)
                        - power-of-10 values with SI prefixes (e.g., kB)

                 1024   Compatibility mode (default). To avoid breaking old scripts:

                        - power-of-2 values with SI prefixes
                        - power-of-10 values with IEC prefixes

          See bs for more details on input parameters.

          Outputs always use correct prefixes. Most outputs include both side-by-side, like:

                 bw=2383.3kB/s (2327.4KiB/s)

          If only one value is reported, then kb_base selects the one to use:

                 1000 -- SI prefixes
                 1024 -- IEC prefixes

   unit_base=int
          Base unit for reporting. Allowed values are:

                 0      Use auto-detection (default).

                 8      Byte based.

                 1      Bit based.

Job description

   name=str
          ASCII  name  of  the  job. This may be used to override the name printed by fio for this job. Otherwise the job name is used. On the command line this parameter has the special
          purpose of also signaling the start of a new job.

   description=str
          Text description of the job. Doesn't do anything except dump this text description when this job is run. It's not parsed.

   loops=int
          Run the specified number of iterations of this job. Used to repeat the same workload a given number of times. Defaults to 1.

   numjobs=int
          Create the specified number of clones of this job. Each clone of job is spawned as an independent thread or process. May be used to setup a larger number  of  threads/processes
          doing  the  same thing. Each thread is reported separately; to see statistics for all clones as a whole, use group_reporting in conjunction with new_group.  See --max-jobs. De‐
          fault: 1.

Time related parameters

   runtime=time
          Limit runtime. The test will run until it completes the configured I/O workload or until it has run for this specified amount of time, whichever occurs first. It can  be  quite
          hard  to  determine  for  how long a specified job will run, so this parameter is handy to cap the total runtime to a given time.  When the unit is omitted, the value is inter‐
          preted in seconds.

   time_based
          If set, fio will run for the duration of the runtime specified even if the file(s) are completely read or written. It will simply loop over the same workload as many  times  as
          the runtime allows.

   startdelay=irange(int)
          Delay  the  start  of  job  for  the specified amount of time. Can be a single value or a range. When given as a range, each thread will choose a value randomly from within the
          range. Value is in seconds if a unit is omitted.

   ramp_time=time
          If set, fio will run the specified workload for this amount of time before logging any performance numbers. Useful for letting performance settle before logging  results,  thus
          minimizing  the  runtime required for stable results. Note that the ramp_time is considered lead in time for a job, thus it will increase the total runtime if a special timeout
          or runtime is specified. When the unit is omitted, the value is given in seconds.

   clocksource=str
          Use the given clocksource as the base of timing. The supported options are:

                 gettimeofday
                        gettimeofday(2)

                 clock_gettime
                        clock_gettime(2)

                 cpu    Internal CPU clock source

          cpu is the preferred clocksource if it is reliable, as it is very fast (and fio is heavy on time calls). Fio will automatically use this clocksource if it's supported and  con‐
          sidered reliable on the system it is running on, unless another clocksource is specifically set. For x86/x86-64 CPUs, this means supporting TSC Invariant.

   gtod_reduce=bool
          Enable  all of the gettimeofday(2) reducing options (disable_clat, disable_slat, disable_bw_measurement) plus reduce precision of the timeout somewhat to really shrink the get‐
          timeofday(2) call count. With this option enabled, we only do about 0.4% of the gettimeofday(2) calls we would have done if all time keeping was enabled.

   gtod_cpu=int
          Sometimes it's cheaper to dedicate a single thread of execution to just getting the current time. Fio (and databases, for instance) are very intensive on gettimeofday(2) calls.
          With this option, you can set one CPU aside for doing nothing but logging current time to a shared memory location. Then the other threads/processes that run I/O workloads need
          only copy that segment, instead of entering the kernel with a gettimeofday(2) call. The CPU set aside for doing these time calls will be excluded from other uses. Fio will man‐
          ually clear it from the CPU mask of other jobs.

   job_start_clock_id=int
          The clock_id passed to the call to clock_gettime used to record job_start in the json output format. Default is 0, or CLOCK_REALTIME.

Target file/device

   directory=str
          Prefix filenames with this directory. Used to place files in a different location than `./'. You can specify a number of directories by separating the names with a ':'  charac‐
          ter.  These directories will be assigned equally distributed to job clones created by numjobs as long as they are using generated filenames. If specific filename(s) are set fio
          will use the first listed directory, and thereby matching the filename semantic (which generates a file for each clone if not specified, but lets all clones use the  same  file
          if set).

          See the filename option for information on how to escape ':' characters within the directory path itself.

          Note: To control the directory fio will use for internal state files use --aux-path.

   filename=str
          Fio  normally  makes  up  a filename based on the job name, thread number, and file number (see filename_format). If you want to share files between threads in a job or several
          jobs with fixed file paths, specify a filename for each of them to override the default. If the ioengine is file based, you can specify a number  of  files  by  separating  the
          names  with  a  ':'  colon.  So if you wanted a job to open `/dev/sda' and `/dev/sdb' as the two working files, you would use `filename=/dev/sda:/dev/sdb'. This also means that
          whenever this option is specified, nrfiles is ignored. The size of regular files specified by this option will be size divided by number of files unless  an  explicit  size  is
          specified by filesize.

          Each  colon  in the wanted path must be escaped with a '\' character. For instance, if the path is `/dev/dsk/foo@3,0:c' then you would use `filename=/dev/dsk/foo@3,0\:c' and if
          the path is `F:\filename' then you would use `filename=F\:\filename'.

          On Windows, disk devices are accessed as `\\.\PhysicalDrive0' for the first device, `\\.\PhysicalDrive1' for the second etc.  Note: Windows and FreeBSD prevent write access  to
          areas of the disk containing in-use data (e.g. filesystems).

          For HTTP and S3 access, specify a valid URL path or S3 key, respectively.  A filename for path-style S3 includes a bucket name (`/bucket/k/e.y') while a virtual-hosted-style S3
          filename (`/k/e.y') does not because its bucket name is specified in http_host.

          The filename `-' is a reserved name, meaning *stdin* or *stdout*. Which of the two depends on the read/write direction set.

   filename_format=str
          If  sharing  multiple files between jobs, it is usually necessary to have fio generate the exact names that you want. By default, fio will name a file based on the default file
          format specification of `jobname.jobnumber.filenumber'. With this option, that can be customized. Fio will recognize and replace the following keywords in this string:

                 $jobname
                        The name of the worker thread or process.

                 $clientuid
                        IP of the fio process when using client/server mode.

                 $jobnum
                        The incremental number of the worker thread or process.

                 $filenum
                        The incremental number of the file for that worker thread or process.

          To have dependent jobs share a set of files, this option can be set to have fio generate filenames that are shared between the two. For  instance,  if  `testfiles.$filenum'  is
          specified, file number 4 for any job will be named `testfiles.4'. The default of `$jobname.$jobnum.$filenum' will be used if no other format specifier is given.

          If you specify a path then the directories will be created up to the main directory for the file.  So for example if you specify `a/b/c/$jobnum` then the directories a/b/c will
          be created before the file setup part of the job.  If you specify directory then the path will be relative that directory, otherwise it is treated as the absolute path.

   unique_filename=bool
          To  avoid collisions between networked clients, fio defaults to prefixing any generated filenames (with a directory specified) with the source of the client connecting. To dis‐
          able this behavior, set this option to 0.

   opendir=str
          Recursively open any files below directory str. This accepts only a single directory and unlike related options, colons appearing in the path must not be escaped.

   lockfile=str
          Fio defaults to not locking any files before it does I/O to them. If a file or file descriptor is shared, fio can serialize I/O to that file to make the end result  consistent.
          This is usual for emulating real workloads that share files. The lock modes are:

                 none   No locking. The default.

                 exclusive
                        Only one thread or process may do I/O at a time, excluding all others.

                 readwrite
                        Read-write locking on the file. Many readers may access the file at the same time, but writes get exclusive access.

   nrfiles=int
          Number  of  files  to  use for this job. Defaults to 1. The size of files will be size divided by this unless explicit size is specified by filesize. Files are created for each
          thread separately, and each file will have a file number within its name by default, as explained in filename section.

   openfiles=int
          Number of files to keep open at the same time. Defaults to the same as nrfiles, can be set smaller to limit the number simultaneous opens.

   file_service_type=str
          Defines how fio decides which file from a job to service next. The following types are defined:

                 random Choose a file at random.

                 roundrobin
                        Round robin over opened files. This is the default.

                 sequential
                        Finish one file before moving on to the next. Multiple files can still be open depending on openfiles.

                 zipf   Use a Zipf distribution to decide what file to access.

                 pareto Use a Pareto distribution to decide what file to access.

                 normal Use a Gaussian (normal) distribution to decide what file to access.

                 gauss  Alias for normal.

          For random, roundrobin, and sequential, a postfix can be appended to tell fio how many I/Os to issue before  switching  to  a  new  file.  For  example,  specifying  `file_ser‐
          vice_type=random:8' would cause fio to issue 8 I/Os before selecting a new file at random. For the non-uniform distributions, a floating point postfix can be given to influence
          how the distribution is skewed. See random_distribution for a description of how that would work.

   ioscheduler=str
          Attempt  to switch the device hosting the file to the specified I/O scheduler before running. If the file is a pipe, a character device file or if device hosting the file could
          not be determined, this option is ignored.

   create_serialize=bool
          If true, serialize the file creation for the jobs. This may be handy to avoid interleaving of data files, which may greatly depend on the filesystem used and even the number of
          processors in the system. Default: true.

   create_fsync=bool
          fsync(2) the data file after creation. This is the default.

   create_on_open=bool
          If true, don't pre-create files but allow the job's open() to create a file when it's time to do I/O. Default: false -- pre-create all necessary files when the job starts.

   create_only=bool
          If true, fio will only run the setup phase of the job. If files need to be laid out or updated on disk, only that will be done -- the actual job contents are not executed.  De‐
          fault: false.

   allow_file_create=bool
          If  true,  fio is permitted to create files as part of its workload. If this option is false, then fio will error out if the files it needs to use don't already exist. Default:
          true.

   allow_mounted_write=bool
          If this isn't set, fio will abort jobs that are destructive (e.g. that write) to what appears to be a mounted device or partition. This should help catch creating inadvertently
          destructive tests, not realizing that the test will destroy data on the mounted file system. Note that some platforms don't allow writing against a mounted device regardless of
          this option. Default: false.

   pre_read=bool
          If this is given, files will be pre-read into memory before starting the given I/O operation. This will also clear the invalidate flag, since it is pointless  to  pre-read  and
          then  drop  the  cache. This will only work for I/O engines that are seek-able, since they allow you to read the same data multiple times. Thus it will not work on non-seekable
          I/O engines (e.g. network, splice). Default: false.

   unlink=bool
          Unlink (delete) the job files when done. Not the default, as repeated runs of that job would then waste time recreating the file set again and again. Default: false.

   unlink_each_loop=bool
          Unlink (delete) job files after each iteration or loop. Default: false.

   zonemode=str
          Accepted values are:

                 none   The zonerange, zonesize zonecapacity and zoneskip parameters are ignored.

                 strided
                        I/O happens in a single zone until zonesize bytes have been transferred.  After that number of bytes has been transferred processing of the next zone starts.  The
                        zonecapacity parameter is ignored.

                 zbd    Zoned  block  device  mode.  I/O happens sequentially in each zone, even if random I/O has been selected. Random I/O happens across all zones instead of being re‐
                        stricted to a single zone.  Trim is handled using a zone reset operation. Trim only considers non-empty sequential write required and sequential  write  preferred
                        zones.

   zonerange=int
          For zonemode=strided, this is the size of a single zone. See also zonesize and zoneskip.

          For zonemode=zbd, this parameter is ignored.

   zonesize=int
          For zonemode=strided, this is the number of bytes to transfer before skipping zoneskip bytes. If this parameter is smaller than zonerange then only a fraction of each zone with
          zonerange bytes will be accessed.  If this parameter is larger than zonerange then each zone will be accessed multiple times before skipping to the next zone.

          For zonemode=zbd, this is the size of a single zone. The zonerange parameter is ignored in this mode. For a job accessing a zoned block device, the specified zonesize must be 0
          or equal to the device zone size. For a regular block device or file, the specified zonesize must be at least 512B.

   zonecapacity=int
          For  zonemode=zbd,  this  defines  the  capacity  of  a  single  zone, which is the accessible area starting from the zone start address. This parameter only applies when using
          zonemode=zbd in combination with regular block devices.  If not specified it defaults to the zone size. If the target device is a zoned block device, the zone capacity  is  ob‐
          tained from the device information and this option is ignored.

   zoneskip=int[z]
          For zonemode=strided, the number of bytes to skip after zonesize bytes of data have been transferred.

          For  zonemode=zbd,  the zonesize aligned number of bytes to skip once a zone is fully written (write workloads) or all written data in the zone have been read (read workloads).
          This parameter is valid only for sequential workloads and ignored for random workloads. For read workloads, see also read_beyond_wp.

   read_beyond_wp=bool
          This parameter applies to zonemode=zbd only.

          Zoned block devices are block devices that consist of multiple zones. Each zone has a type, e.g. conventional or sequential. A conventional zone can be written  at  any  offset
          that  is a multiple of the block size. Sequential zones must be written sequentially. The position at which a write must occur is called the write pointer. A zoned block device
          can be either host managed or host aware. For host managed devices the host must ensure that writes happen sequentially. Fio recognizes  host  managed  devices  and  serializes
          writes to sequential zones for these devices.

          If  a  read occurs in a sequential zone beyond the write pointer then the zoned block device will complete the read without reading any data from the storage medium. Since such
          reads lead to unrealistically high bandwidth and IOPS numbers fio only reads beyond the write pointer if explicitly told to do so. Default: false.

   max_open_zones=int
          When a zone of a zoned block device is partially written (i.e. not all sectors of the zone have been written), the zone is in one of three  conditions:  'implicit  open',  'ex‐
          plicit  open'  or 'closed'. Zoned block devices may have a limit called 'max_open_zones' (same name as the parameter) on the total number of zones that can simultaneously be in
          the 'implicit open' or 'explicit open' conditions. Zoned block devices may have another limit called 'max_active_zones', on the total number of zones that can simultaneously be
          in the three conditions. The max_open_zones parameter limits the number of zones to which write commands are issued by all fio jobs, that is, limits the number  of  zones  that
          will be in the conditions. When the device has the max_open_zones limit and does not have the max_active_zones limit, the max_open_zones parameter limits the number of zones in
          the  two  open  conditions  up  to  the  limit. In this case, fio includes zones in the two open conditions to the write target zones at fio start. When the device has both the
          max_open_zones and the max_active_zones limits, the max_open_zones parameter limits the number of zones in the three conditions up to the limit.  In  this  case,  fio  includes
          zones in the three conditions to the write target zones at fio start.

          This  parameter  is relevant only if the zonemode=zbd is used. The default value is always equal to the max_open_zones limit of the target zoned block device and a value higher
          than this limit cannot be specified by users unless the option ignore_zone_limits is specified. When ignore_zone_limits is specified or the target  device  does  not  have  the
          max_open_zones limit, max_open_zones can specify 0 to disable any limit on the number of zones that can be simultaneously written to by all jobs.

   job_max_open_zones=int
          In  the  same  manner as max_open_zones, limit the number of open zones per fio job, that is, the number of zones that a single job can simultaneously write to. A value of zero
          indicates no limit. Default: zero.

   ignore_zone_limits=bool
          If this option is used, fio will ignore the maximum number of open zones limit of the zoned block device in use, thus allowing the option max_open_zones value to be larger than
          the device reported limit. Default: false.

   zone_reset_threshold=float
          A number between zero and one that indicates the ratio of written bytes in the zones with write pointers in the IO range to the size of the IO  range.  When  current  ratio  is
          above  this  ratio, zones are reset periodically as zone_reset_frequency specifies. If there are multiple jobs when using this option, the IO range for all write jobs has to be
          the same.

   zone_reset_frequency=float
          A number between zero and one that indicates how often a zone reset should be issued if the zone reset threshold has been exceeded. A zone reset is submitted after  each  (1  /
          zone_reset_frequency) write requests. This and the previous parameter can be used to simulate garbage collection activity.

I/O type

   direct=bool
          If value is true, use non-buffered I/O. This is usually O_DIRECT. Note that OpenBSD and ZFS on Solaris don't support direct I/O. On Windows the synchronous ioengines don't sup‐
          port direct I/O. Default: false.

   buffered=bool
          If value is true, use buffered I/O. This is the opposite of the direct option. Defaults to true.

   readwrite=str, rw=str
          Type of I/O pattern. Accepted values are:

                 read   Sequential reads.

                 write  Sequential writes.

                 trim   Sequential trims (Linux block devices and SCSI character devices only).

                 randread
                        Random reads.

                 randwrite
                        Random writes.

                 randtrim
                        Random trims (Linux block devices and SCSI character devices only).

                 rw,readwrite
                        Sequential mixed reads and writes.

                 randrw Random mixed reads and writes.

                 trimwrite
                        Sequential trim+write sequences. Blocks will be trimmed first, then the same blocks will be written to. So if `io_size=64K' is specified, Fio will trim a total of
                        64K  bytes  and also write 64K bytes on the same trimmed blocks. This behaviour will be consistent with `number_ios' or other Fio options limiting the total bytes
                        or number of I/O's.

                 randtrimwrite
                        Like trimwrite , but uses random offsets rather than sequential writes.

          Fio defaults to read if the option is not specified. For the mixed I/O types, the default is to split them 50/50. For certain types of I/O the result may still be skewed a bit,
          since the speed may be different.

          It is possible to specify the number of I/Os to do before getting a new offset by appending `:<nr>' to the end of the string given. For  a  random  read,  it  would  look  like
          `rw=randread:8'  for  passing in an offset modifier with a value of 8. If the suffix is used with a sequential I/O pattern, then the `<nr>' value specified will be added to the
          generated offset for each I/O turning sequential I/O into sequential I/O with holes.  For instance, using `rw=write:4k' will skip 4k for every write. Also see the  rw_sequencer
          option. If this is used with verify then verify_header_seed option will be disabled, unless its explicitly enabled.

   rw_sequencer=str
          If  an  offset  modifier is given by appending a number to the `rw=str' line, then this option controls how that number modifies the I/O offset being generated. Accepted values
          are:

                 sequential
                        Generate sequential offset.

                 identical
                        Generate the same offset.

          sequential is only useful for random I/O, where fio would normally generate a new random offset for every I/O. If you append e.g. 8 to randread, i.e. `rw=randread:8' you  would
          get a new random offset for every 8 I/Os. The result would be a sequence of 8 sequential offsets with a random starting point.  However this behavior may change if a sequential
          I/O  reaches  end  of the file. As sequential I/O is already sequential, setting sequential for that would not result in any difference. identical behaves in a similar fashion,
          except it sends the same offset 8 number of times before generating a new offset.

          Example #1:

                 rw=randread:8
                 rw_sequencer=sequential
                 bs=4k

          The generated sequence of offsets will look like this: 4k, 8k, 12k, 16k, 20k, 24k, 28k, 32k, 92k, 96k, 100k, 104k, 108k, 112k, 116k, 120k, 48k, 52k ...

          Example #2:

                 rw=randread:8
                 rw_sequencer=identical
                 bs=4k

          The generated sequence of offsets will look like this: 4k, 4k, 4k, 4k, 4k, 4k, 4k, 4k, 92k, 92k, 92k, 92k, 92k, 92k, 92k, 92k, 48k, 48k, 48k ...

   unified_rw_reporting=str
          Fio normally reports statistics on a per data direction basis, meaning that reads, writes, and trims are accounted and reported separately. This option determines  whether  fio
          reports the results normally, summed together, or as both options.  Accepted values are:

          none   Normal statistics reporting.

          mixed  Statistics are summed per data direction and reported together.

          both   Statistics are reported normally, followed by the mixed statistics.

          0      Backward-compatible alias for none.

          1      Backward-compatible alias for mixed.

          2      Alias for both.

   randrepeat=bool
          Seed all random number generators in a predictable way so the pattern is repeatable across runs. Default: true.

   allrandrepeat=bool
          Alias for randrepeat. Default: true.

   randseed=int
          Seed  the  random  number  generators based on this seed value, to be able to control what sequence of output is being generated. If not set, the random sequence depends on the
          randrepeat setting.

   fallocate=str
          Whether pre-allocation is performed when laying down files.  Accepted values are:

                 none   Do not pre-allocate space.

                 native Use a platform's native pre-allocation call but fall back to none behavior if it fails/is not implemented.

                 posix  Pre-allocate via posix_fallocate(3).

                 keep   Pre-allocate via fallocate(2) with FALLOC_FL_KEEP_SIZE set.

                 truncate
                        Extend file to final size using ftruncate|(2) instead of allocating.

                 0      Backward-compatible alias for none.

                 1      Backward-compatible alias for posix.

          May not be available on all supported platforms. keep is only available on Linux. If using ZFS on Solaris this cannot be set to posix because ZFS  doesn't  support  pre-alloca‐
          tion. Default: native if any pre-allocation methods except truncate are available, none if not.

          Note  that using truncate on Windows will interact surprisingly with non-sequential write patterns. When writing to a file that has been extended by setting the end-of-file in‐
          formation, Windows will backfill the unwritten portion of the file up to that offset with zeroes before issuing the new write. This means that a single small write to  the  end
          of an extended file will stall until the entire file has been filled with zeroes.

   fadvise_hint=str
          Use posix_fadvise(2) or posix_madvise(2) to advise the kernel what I/O patterns are likely to be issued. Accepted values are:

                 0      Backwards compatible hint for "no hint".

                 1      Backwards compatible hint for "advise with fio workload type". This uses FADV_RANDOM for a random workload, and FADV_SEQUENTIAL for a sequential workload.

                 sequential
                        Advise using FADV_SEQUENTIAL.

                 random Advise using FADV_RANDOM.

                 noreuse
                        Advise  using  FADV_NOREUSE.  This  may be a no-op on older Linux kernels. Since Linux 6.3, it provides a hint to the LRU algorithm.  See the posix_fadvise(2) man
                        page.

   write_hint=str
          Use fcntl(2) to advise the kernel what life time to expect from a write. Only supported on Linux, as of version 4.13. Accepted values are:

                 none   No particular life time associated with this file.

                 short  Data written to this file has a short life time.

                 medium Data written to this file has a medium life time.

                 long   Data written to this file has a long life time.

                 extreme
                        Data written to this file has a very long life time.

          The values are all relative to each other, and no absolute meaning should be associated with them.

   offset=int[%|z]
          Start I/O at the provided offset in the file, given as either a fixed size in bytes, zones or a percentage. If a percentage is given, the generated offset will  be  aligned  to
          the  minimum  blocksize  or to the value of offset_align if provided. Data before the given offset will not be touched. This effectively caps the file size at `real_size - off‐
          set'. Can be combined with size to constrain the start and end range of the I/O workload.  A percentage can be specified by a number between 1 and 100 followed by '%', for  ex‐
          ample, `offset=20%' to specify 20%. In ZBD mode, value can be set as number of zones using 'z'.

   offset_align=int
          If  set  to  non-zero value, the byte offset generated by a percentage offset is aligned upwards to this value. Defaults to 0 meaning that a percentage offset is aligned to the
          minimum block size.

   offset_increment=int[%|z]
          If this is provided, then the real offset becomes `offset + offset_increment * thread_number', where the thread number is a counter that starts at 0 and is incremented for each
          sub-job (i.e. when numjobs option is specified). This option is useful if there are several jobs which are intended to operate on a file in  parallel  disjoint  segments,  with
          even  spacing  between the starting points. Percentages can be used for this option.  If a percentage is given, the generated offset will be aligned to the minimum blocksize or
          to the value of offset_align if provided.In ZBD mode, value can be set as number of zones using 'z'.

   number_ios=int
          Fio will normally perform I/Os until it has exhausted the size of the region set by size, or if it exhaust the allocated time (or hits an error condition). With  this  setting,
          the  range/size can be set independently of the number of I/Os to perform. When fio reaches this number, it will exit normally and report status. Note that this does not extend
          the amount of I/O that will be done, it will only stop fio if this condition is met before other end-of-job criteria.

   fsync=int
          If writing to a file, issue an fsync(2) (or its equivalent) of the dirty data for every number of blocks given. For example, if you give 32 as a parameter, fio  will  sync  the
          file after every 32 writes issued. If fio is using non-buffered I/O, we may not sync the file. The exception is the sg I/O engine, which synchronizes the disk cache anyway. De‐
          faults to 0, which means fio does not periodically issue and wait for a sync to complete. Also see end_fsync and fsync_on_close.

   fdatasync=int
          Like  fsync but uses fdatasync(2) to only sync data and not metadata blocks. In Windows, DragonFlyBSD or OSX there is no fdatasync(2) so this falls back to using fsync(2).  De‐
          faults to 0, which means fio does not periodically issue and wait for a data-only sync to complete.

   write_barrier=int
          Make every N-th write a barrier write.

   sync_file_range=str:int
          Use sync_file_range(2) for every int number of write operations. Fio will track range of writes that have happened since the last sync_file_range(2) call. str can currently  be
          one or more of:

                 wait_before
                        SYNC_FILE_RANGE_WAIT_BEFORE

                 write  SYNC_FILE_RANGE_WRITE

                 wait_after
                        SYNC_FILE_RANGE_WRITE_AFTER

          So  if you do `sync_file_range=wait_before,write:8', fio would use `SYNC_FILE_RANGE_WAIT_BEFORE | SYNC_FILE_RANGE_WRITE' for every 8 writes. Also see the sync_file_range(2) man
          page. This option is Linux specific.

   overwrite=bool
          If true, writes to a file will always overwrite existing data. If the file doesn't already exist, it will be created before the write phase begins. If the file  exists  and  is
          large enough for the specified write phase, nothing will be done. Default: false.

   end_fsync=bool
          If true, fsync(2) file contents when a write stage has completed.  Default: false.

   fsync_on_close=bool
          If true, fio will fsync(2) a dirty file on close. This differs from end_fsync in that it will happen on every file close, not just at the end of the job. Default: false.

   rwmixread=int
          Percentage of a mixed workload that should be reads. Default: 50.

   rwmixwrite=int
          Percentage  of  a  mixed  workload  that should be writes. If both rwmixread and rwmixwrite is given and the values do not add up to 100%, the latter of the two will be used to
          override the first. This may interfere with a given rate setting, if fio is asked to limit reads or writes to a certain rate. If that is the case, then the distribution may  be
          skewed. Default: 50.

   random_distribution=str:float[:float][,str:float][,str:float]
          By  default,  fio will use a completely uniform random distribution when asked to perform random I/O. Sometimes it is useful to skew the distribution in specific ways, ensuring
          that some parts of the data is more hot than others.  fio includes the following distribution models:

                 random Uniform random distribution

                 zipf   Zipf distribution

                 pareto Pareto distribution

                 normal Normal (Gaussian) distribution

                 zoned  Zoned random distribution zoned_abs Zoned absolute random distribution

          When using a zipf or pareto distribution, an input value is also needed to define the access pattern. For zipf, this is the `Zipf theta'.  For pareto, it's the `Pareto  power'.
          Fio  includes  a test program, fio-genzipf, that can be used visualize what the given input values will yield in terms of hit rates. If you wanted to use zipf with a `theta' of
          1.2, you would use `random_distribution=zipf:1.2' as the option. If a non-uniform model is used, fio will disable use of the random map. For the normal distribution,  a  normal
          (Gaussian) deviation is supplied as a value between 0 and 100.

          The  second,  optional float is allowed for pareto, zipf and normal distributions. It allows one to set base of distribution in non-default place, giving more control over most
          probable outcome. This value is in range [0-1] which maps linearly to range of possible random values.  Defaults are: random for pareto and zipf, and 0.5 for  normal.   If  you
          wanted to use zipf with a `theta` of 1.2 centered on 1/4 of allowed value range, you would use `random_distribution=zipf:1.2:0.25`.

          For a zoned distribution, fio supports specifying percentages of I/O access that should fall within what range of the file or device. For example, given a criteria of:

                 60% of accesses should be to the first 10%
                 30% of accesses should be to the next 20%
                 8% of accesses should be to the next 30%
                 2% of accesses should be to the next 40%

          we can define that through zoning of the random accesses. For the above example, the user would do:

                 random_distribution=zoned:60/10:30/20:8/30:2/40

          A  zoned_abs distribution works exactly like thezoned, except that it takes absolute sizes. For example, let's say you wanted to define access according to the following crite‐
          ria:

                 60% of accesses should be to the first 20G
                 30% of accesses should be to the next 100G
                 10% of accesses should be to the next 500G

          we can define an absolute zoning distribution with:

                 random_distribution=zoned:60/10:30/20:8/30:2/40

          For both zoned and zoned_abs, fio supports defining up to 256 separate zones.

          Similarly to how bssplit works for setting ranges and percentages of block sizes. Like bssplit, it's possible to specify separate zones for reads, writes, and  trims.  If  just
          one set is given, it'll apply to all of them.

   percentage_random=int[,int][,int]
          For  a random workload, set how big a percentage should be random. This defaults to 100%, in which case the workload is fully random. It can be set from anywhere from 0 to 100.
          Setting it to 0 would make the workload fully sequential. Any setting in between will result in a random mix of sequential and random I/O, at the given percentages. Comma-sepa‐
          rated values may be specified for reads, writes, and trims as described in blocksize.

   norandommap
          Normally fio will cover every block of the file when doing random I/O. If this option is given, fio will just get a new random offset without looking at past I/O history.  This
          means  that  some blocks may not be read or written, and that some blocks may be read/written more than once. If this option is used with verify then verify_header_seed will be
          disabled. If this option is used with verify and multiple blocksizes (via bsrange), only intact blocks are verified, i.e., partially-overwritten blocks  are  ignored.  With  an
          async I/O engine and an I/O depth > 1, header write sequence number verification will be disabled. See verify_write_sequence.

   softrandommap=bool
          See  norandommap. If fio runs with the random block map enabled and it fails to allocate the map, if this option is set it will continue without a random block map. As coverage
          will not be as complete as with random maps, this option is disabled by default.

   random_generator=str
          Fio supports the following engines for generating I/O offsets for random I/O:

                 tausworthe
                        Strong 2^88 cycle random number generator.

                 lfsr   Linear feedback shift register generator.

                 tausworthe64
                        Strong 64-bit 2^258 cycle random number generator.

          tausworthe is a strong random number generator, but it requires tracking on the side if we want to ensure that blocks are only read or written once.  lfsr  guarantees  that  we
          never  generate  the  same  offset  twice,  and it's also less computationally expensive. It's not a true random generator, however, though for I/O purposes it's typically good
          enough. lfsr only works with single block sizes, not with workloads that use multiple block sizes. If used with such a workload, fio may read  or  write  some  blocks  multiple
          times. The default value is tausworthe, unless the required space exceeds 2^32 blocks. If it does, then tausworthe64 is selected automatically.