Using Iostat to monitor I/O status under Linux

We can use SAR (1), Pidstat (1), Mpstat (1), Vmstat (8) to monitor

First, installation

Yum Install Sysstat

Second, the parameter explanation

FILES

/proc/stat contains system statistics./proc/uptime contains system uptime./proc/partitions contains disk statistics (for Pre 2.5 kernels that has been patched)./proc/diskstats contains disks statistics (for post 2.5 kernels)./sys contains STA Tistics for block devices (post 2.5 kernels)./proc/self/mountstats contains statistics for network Filesystems./dev/disk c Ontains Persistent device names.

CPU Utilization Report

%user    Show  the  percentage of  CPU utilization that occurred and executing at the user level (Applica-tio n).%nice   Show The percentage of CPU utilization that occurred and executing at the user level with  nicep Riority.%system    Show The  percentage of CPU utilization that occurred while executing at the system level (kernel). %iowait Show The percentage of time that the    CPU or CPUs were idle during which the system had an outstand-ing disk I /o request.%steal Show The percentage of time    spent in involuntary wait by the virtual CPU or CPUs while the hypervi- SOR is servicing another virtual Processor.%idle Show the percentage of time that the    CPU or CPUs were idle and the System do not has an outstand-ing disk I/O request.

Device Utilization Report

TPS indicate the number of transfers per second that were issued to the device. A transfer is an I/O request to the device. Multiple logical requests can is combined into a single I/O request to the device. A transfer is of indeterminate size. blk_read/s indicate the amount of data read from the device expressed in a number of blocks per second.  Blocks is equivalent to sectors with kernels 2.4 and later and therefore has a size of $ bytes. With older kernels, a block is of indeterminate size. BLK_WRTN/S indicate the amount of data written to the device expressed in a number of blocks per second. Blk_read the total number of blocks read. Blk_wrtn the total number of blocks written.kb_read/s indicate the amount of data read from the device expressed in Kilobytes per second.kb_wrtn/s indicate the amount of data written to the device expressed in kilobytes per Second.kb_r EAD The total number of kilobytes Read.kb_wrtn the total number ofKilobytes written. mb_read/s indicate the amount of data read from the device expressed in megabytes per second. MB_WRTN/S indicate the amount of data written to the device expressed in megabytes per second. Mb_read the total number of megabytes read. Mb_wrtn the total number of megabytes written.rrqm/s the number of read requests merged per second that were queued  To the device.wrqm/s the number of write requests merged per second this were queued to the device.r/s the number of Read requests that were issued to the device per second.w/s the number of write requests this were issued to the Devic E per second.rsec/s The number of sectors read from the device per second.wsec/s the number of sectors written to th E device per second.rkb/s the number of kilobytes read from the device per second.wkb/s the number of kilobytes writ Ten to the device per second.rmb/s the number of megabytes read from the device per second.wmb/s the number of Megab Ytes WritteN to the device per Second.avgrq-sz the average size (in sectors) of the requests, were issued to the device.avgqu- SZ the average queue length of the requests that were issued to the device.await the average time (in Millisecon DS) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing THEM.SVCTM the average service time  (in milliseconds) for I/O requests that were issued to the device. warning! Don't trust this field any more. This field would be removed in a future Sysstat version.%util Percentage of CPU time during which I/O requests were is Sued to the device (bandwidth utilization for the device). Device saturation occurs when the this value was close to 100%.

 network Filesystem report

RBLK_NOR/S indicate the number of blocks read by applications via the Read (2) system, call interface. A block has a size of up to bytes.wblk_nor/s indicate the number of blocks written by applications via the Write (2) Syst    EM call interface.rblk_dir/s indicate the number of blocks read from files opened with the O_direct flag.wblk_dir/s Indicate the number of blocks written to files opened with the O_direct flag.rblk_svr/s indicate the number of blocks R EAD from the server by the NFS client via a NFS READ request.wblk_svr/s indicate the number of blocks written to the S Erver by the NFS client via a NFS WRITE request.rkb_nor/s indicate the number of kilobytes read by applications via th e Read (2) system call interface.wkb_nor/s indicate the number of kilobytes written by applications via the Write (2) sys    TEM call INTERFACE.RKB_DIR/S indicate the number of kilobytes read from files opened with the O_direct flag.wkb_dir/s Indicate the number of kilobytes WritteN to files opened with the O_direct flag.rkb_svr/s indicate the number of kilobytes read from the server by the NFS CLI ENT via an NFS READ request.wkb_svr/s indicate the number of kilobytes written to the server by the NFS client via an NFS WRITE request.rmb_nor/s indicate the number of megabytes read by applications via the Read (2) System call interface    . wmb_nor/s indicate the number of megabytes written by applications via the Write (2) system call INTERFACE.RMB_DIR/S Indicate the number of megabytes read from files opened with the O_direct flag.wmb_dir/s indicate the number of Megaby TES written to files opened with the O_direct flag.rmb_svr/s indicate the number of megabytes read from the server by T He NFS client via an NFS READ request.wmb_svr/s indicate the number of megabytes written to the server by the NFS Clien T via an NFS WRITE request.ops/s indicate the number of operations this were issued to the filesystem per second.rops/ s indicate the number of ' read ' operations that were issued to the filesystem per second.wops/s indicate the number of ' write ' operations th At were issued to the filesystem per second.

　Command parameters

-C Display CPU Usage-D displays disk usage-m displays the-N display in units of m in meters-C displays disk array (LVM) information-n Displays NFS usage-p[disk] Displays disk and partition conditions-T displays information about the terminal and the CPU-x display details-V Display version Information

Third, the mode of use Device Utilization Report

[[email protected] ~]# iostat-d-K 2 4Linux 2.6.32-431.11.2.el6.x86_64 (localhost) 01/08/2016 _x86_64_ (4 CPU) Device:tps kb_read/s kb_wrtn/s kb_read Kb_wrtnxvda 114.22 13.80 713.21 37 2234513 19241775188xvdb 19.26 21.24 144.25 573067009 3891812336device:tps kb_read              /s KB_WRTN/S kb_read Kb_wrtnxvda 0.00 0.00 0.00 0 0xvdb 4.50 0.00 18.00 0 36device:tps kb_read/s kb_wrtn/s kb_read kb_w          Rtnxvda 0.00 0.00 0.00 0 0xvdb 10.50 0.00 42.00         0 84device:tps kb_read/s kb_wrtn/s kb_read Kb_wrtnxvda 0.00 0.00 0.00 0 0xvdb 20.50 0.00 86.00 0 172

Parameter description

• TPS: The number of times the device is transferred per second, multiple logical requests can be combined into a single I/O request device. The transport has an indeterminate size.

• KB_READ/S: The amount of data read from the device per second

• KB_WRTN/S: The amount of data written to the device per second

• Kb_read: Total amount of data read

• KB_WRTN: Total amount of data written

[[email protected] ~]# iostat-d-x-k 1 10Linux 2.6.32-431.11.2.el6.x86_64 (localhost) 01/08/2016 _x86_6 4_ (4 CPU) device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%util              Xvda 0.02 0.69 0.56 113.65 13.80 713.21 12.73 0.00 0.03 0.48 5.53xvdb     0.02 17.88 1.07 18.19 21.24 144.27 17.19 0.13 6.71 1.69 3.25device:rrqm/s wrqm/s     r/s w/s rkb/s wkb/s avgrq-sz avgqu-sz await SVCTM%utilxvda 0.00 0.00 0.00 0.00     0.00 0.00 0.00 0.00 0.00 0.00 0.00XVDB 0.00 0.00 0.00 5.00 0.00 20.00   8.00 0.11 22.00 4.40 2.20device:rrqm/s wrqm/s r/s w/s rkb/s wkb/s avgrq-sz Avgqu-sz Await SVCTM%utilxvda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0 .00xvdb 0.00 0.00 0.00 16.00 0.00 64.00 8.00 0.80 50.19 6.81 10.90 

Total io (IO)/s = r/s (read) +w/s (write)

Average wait time = single I/O service time * (1 + 2 + ... + total requests-1)/Total requests

Parameter description

• RRQM/S: The number of read operations per second for the merge. That is Delta (rmerge)/s

• WRQM/S: The number of write operations per second for the merge. That is, Delta (wmerge)/s

• R/S: Number of Read I/O devices completed per second. Delta (RIO)/s

• W/S: Number of write I/O devices completed per second. Delta (WIO)/s

• RSEC/S: Number of Read sectors per second. Delta (rsect)/s

• WSEC/S: Number of Write sectors per second. Delta (wsect)/s

• rkb/s: Read K bytes per second. Is half of the rsect/s because the size of each sector is 512 bytes. (Calculation required)

• wkb/s: Writes K bytes per second. It's half the wsect/s. (Calculation required)

• AVGRQ-SZ: The data size (sector) of the average Per device I/O operation. Delta (Rsect+wsect)/delta (Rio+wio)

• Avgqu-sz: Average I/O queue length. That is Delta (AVEQ)/s/1000 (because Aveq is in milliseconds).

• Await: The average wait time (in milliseconds) for each device I/O operation. Delta (Ruse+wuse)/delta (Rio+wio)

• SVCTM: The average service time (in milliseconds) per device I/O operation. Delta (use)/delta (RIO+WIO)

• %util: How much time in a second is spent on I/O operations, or how many times in a second I/O queues are non-empty. That is, Delta (use)/s/1000 (because the use is in milliseconds)

If%util is close to 100%, it indicates that there are too many I/O requests, the I/O system is full, and the disk may have bottlenecks.
If idle is less than 70% io pressure is larger, general reading speed has more wait

Avgqu-sz is a need to pay attention to the place, this is the direct operation of the data size, if the number of times, but the data is small, in fact, the IO will be very small. If the data is large, the IO data will be high, via AVGQU-SZX (r/s or w/s) = RSEC/S or wsec/s

CPU Utilization Report

[[email protected] ~]# iostat-c 1 10Linux 2.6.32-431.11.2.el6.x86_64 (localhost)        01/08/2016      _x86_64_        (4 CPU) Avg-cpu:  %user   %nice%system%iowait  %steal   %idle           5.09    0.00    2.08    4.28    0.00   88.55avg-cpu:  %user   %nice%system%iowait%steal%idle           1.75    0.00    0.50    0.00    0.00   97.75avg-cpu:  %user   %nice%system%iowait%steal%idle           1.50    0.00    0.50    0.00    0.00   98.00avg-cpu:  %user   %nice%system%iowait  %steal   %idle           0.25    0.00    0.25    0.00    0.00   99.50

Parameter description

• %user:cpu percentage of time in user mode

• %nice:cpu percentage of time in user mode with nice value

• %system:cpu percentage of time in System mode

• %iowait:cpu the percentage of time to wait for the input output to finish

• %steal: The percentage of the virtual CPU's unconscious wait time when the hypervisor maintains a second virtual processor

• %IDLE:CPU Idle time percentage

If the value of%iowait is too high, indicates that the hard disk has an I/O bottleneck and a high%idle value indicates that the CPU is idle

If the%idle value is high but the system responds slowly, it is possible that the CPU waits for memory allocation, the memory capacity should be increased, if the%idle value lasts below 10, the CPU processing power of the system is relatively low, which indicates that the most necessary resource to be solved in the system is CPU.

Common usage

Iostat-d-K 1        #查看TPS和吞吐量信息iostat-D-x-k 1      #查看设备使用率 (%util), Response time (await) Iostat-c 1            #查看cpu状态

Reference articles

Http://www.cnblogs.com/peida/archive/2012/12/28/2837345.html

Http://www.mjmwired.net/kernel/Documentation/iostats.txt

http://www.orczhou.com/index.php/2010/03/iostat-detail/

Http://www.php-oa.com/2009/02/03/iostat.html

Using Iostat to monitor I/O status under Linux