iostat is a command-line monitoring tool for CPU and block device input/output statistics on Linux, distributed in the sysstat package. In Linux monitoring, it is the standard first look at storage behavior: it reports throughput, request rates, latency, queue depth and utilization for each disk, partition or device-mapper volume.
iostat reads cumulative kernel counters from /proc/diskstats and /proc/stat and converts them into rates over an interval. It does not trace individual requests and adds practically no overhead, which makes it safe to run on busy production systems when an application feels slow and storage is a suspect.
How iostat calculates its numbers
The kernel keeps monotonically increasing counters for each block device: completed reads and writes, sectors transferred, merged requests, milliseconds spent on I/O and the weighted time requests spent in the queue. iostat samples these counters at the start and end of an interval and divides the difference by the elapsed time.
The first report shows averages since boot, which are rarely useful for troubleshooting. Running iostat -x 5 produces a new report every five seconds, and the -y option suppresses the since-boot report entirely so that every line reflects current activity.
The CPU section reports the share of time spent in user, nice, system, iowait, steal and idle states. %steal matters on virtual machines, where it shows time the hypervisor gave to other guests, and %iowait is the share of idle time during which at least one I/O request was outstanding.
Reading the extended device statistics
The extended report produced by `-x` is where most diagnostic value lies. The table explains the most important columns and the typical values that indicate trouble.
| Column | Meaning | What to look for |
|---|---|---|
| r/s, w/s | Completed reads and writes per second | IOPS demand on the device |
| rkB/s, wkB/s | Kilobytes read and written per second | Throughput against device limits |
| rareq-sz, wareq-sz | Average request size in kilobytes | Small values mean random I/O |
| rrqm/s, wrqm/s | Requests merged per second | Merging of adjacent sequential I/O |
| r_await, w_await | Average time per request in milliseconds, including queueing | Latency seen by applications |
| aqu-sz | Average number of requests in flight | Sustained values far above 1 suggest saturation |
| %util | Share of time the device had at least one request | Meaningful mainly for single-queue disks |
Latency must be judged against the device class. The following table gives orientation values for healthy average await under moderate load; actual figures depend on request size, queue depth and controller.
| Device class | Typical average await | Signs of trouble |
|---|---|---|
| NVMe SSD | Well under 1 ms | Several milliseconds sustained |
| SATA SSD | Around 1 ms or less | Tens of milliseconds |
| 7,200 rpm hard disk | Roughly 5 to 15 ms | Over 30 ms with a growing queue |
| Network or cloud block volume | About 1 to several ms | Spikes matching throttling limits |
Why %util misleads on modern storage
%util is not a capacity measure on devices that serve many requests in parallel. It records the share of time with at least one request in flight, so an NVMe drive that handles dozens of requests concurrently can show 100% utilization while having ample headroom.
The same applies to RAID arrays, logical volumes spread across many disks and network block storage. For these devices, await and aqu-sz are better indicators: if latency stays flat while the queue grows, the device is coping; if latency climbs with the queue, it is saturated.
The older svctm column has been removed from current sysstat versions because it was derived from %util and gave equally misleading service-time estimates for parallel devices.
iostat among Linux monitoring tools
iostat answers whether a block device is busy or slow, but not which process causes the load. It is therefore usually combined with neighboring tools in the monitoring topic.
| Tool | Focus | Best question it answers |
|---|---|---|
| iostat | Per-device I/O and CPU summary | Is the disk saturated or slow |
| vmstat | Memory, swap, run queue, overall I/O | Is the whole system under pressure |
| top and htop | Per-process CPU and memory | Which process is consuming resources |
| iotop and pidstat -d | Per-process I/O | Which process generates the I/O |
| Pressure stall information | Time tasks wait on I/O | How much work is delayed by I/O |
| strace and lsof | System calls and open files | What a specific process is doing |
A useful workflow is top-down: vmstat or pressure stall information reveals that I/O delays work, iostat identifies the affected device, and pidstat or iotop names the responsible process.
Useful options and common mistakes
A small set of options covers most practical sessions. -x adds extended statistics, -d limits output to devices, -m or -h makes sizes readable, -z hides idle devices, -t adds timestamps, -p sda includes partitions, and -N shows device-mapper names such as LVM volumes.
- Trusting the first report: it averages since boot and hides the current problem.
- Reading high iowait as a disk problem: iowait can rise simply because the CPU is otherwise idle.
- Treating 100% util as full: parallel devices may still accept more work.
- Using long intervals for bursty loads: a 60-second average can hide 5-second latency spikes.
- Ignoring request size: comparing IOPS between workloads with different request sizes is misleading.
In simple terms
A computer constantly reads from and writes to its disks. iostat is a traffic counter at the entrance of each disk. Every few seconds it reports how many requests passed, how much data moved and how long each request had to wait.
When a program feels slow, these numbers show whether the disk is the bottleneck. Long waits and a growing line of requests mean the disk cannot keep up; short waits mean the cause lies somewhere else.
A practical example
Users of an internal reporting application complain that queries slow down every night around 01:00. The database runs on a Linux virtual machine with a cloud block volume, and the CPU graph shows nothing unusual, so the administrator suspects storage contention.
During the next slow period, iostat -xyzt 5 shows the data volume at about 3,000 writes per second with w_await jumping from 2 ms to over 40 ms and aqu-sz rising above 60. pidstat -d 5 reveals that a backup agent is copying database files at the same time as the reporting queries. The write rate sits exactly at the provisioned IOPS limit of the volume.
The backup window is moved to 04:00 and the volume is upgraded to a higher performance tier. On the following nights, write latency stays near 2 ms and report run times return to normal.