Use iostat in Linux to monitor the I/O status.

Use iostat in Linux to monitor the I/O status.

We can use sar (1), pidstat (1), mpstat (1), vmstat (8) for monitoring.

I. Installation

yum install sysstat

Ii. Parameter explanation

FILES

/proc/stat contains system statistics./proc/uptime contains system uptime./proc/partitions contains disk statistics (for pre 2.5 kernels that have been patched)./proc/diskstats contains disks statistics (for post 2.5 kernels)./sys contains statistics for block devices (post 2.5 kernels)./proc/self/mountstats contains statistics for network filesystems./dev/disk contains persistent device names.

CPU Utilization Report

%user    Show  the  percentage  of CPU utilization that occurred while executing at the user level (applica-tion).%nice   Show the percentage of CPU utilization that occurred while executing at the user  level  with  nicepriority.%system    Show  the percentage of CPU utilization that occurred while executing at the system level (kernel).%iowait    Show the percentage of time that the CPU or CPUs were idle during which the system had an outstand-ing disk I/O request.%steal    Show the percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervi-sor was servicing another virtual processor.%idle    Show the percentage of time that the CPU or CPUs were idle and the system did not have an outstand-ing disk I/O request.

Device Utilization Report

tps   Indicate  the  number  of transfers per second that were issued to the device. A transfer is an I/O      request to the device. Multiple logical requests can be combined into a single I/O request  to  the      device. A transfer is of indeterminate size.Blk_read/s    Indicate the amount of data read from the device expressed in a number of blocks per second. Blocks      are equivalent to sectors with kernels 2.4 and later and therefore have a size of 512  bytes.  With      older kernels, a block is of indeterminate size.Blk_wrtn/s    Indicate the amount of data written to the device expressed in a number of blocks per second.Blk_read    The total number of blocks read.Blk_wrtn    The total number of blocks written.kB_read/s    Indicate the amount of data read from the device expressed in kilobytes per second.kB_wrtn/s    Indicate the amount of data written to the device expressed in kilobytes per second.kB_read    The total number of kilobytes read.kB_wrtn    The total number of kilobytes written.MB_read/s    Indicate the amount of data read from the device expressed in megabytes per second.MB_wrtn/s    Indicate the amount of data written to the device expressed in megabytes per second.MB_read    The total number of megabytes read.MB_wrtn    The total number of megabytes written.rrqm/s    The number of read requests merged per second that were queued to the device.wrqm/s    The number of write requests merged per second that were queued to the device.r/s    The number of read requests that were issued to the device per second.w/s    The number of write requests that were issued to the device per second.rsec/s    The number of sectors read from the device per second.wsec/s    The number of sectors written to the device per second.rkB/s    The number of kilobytes read from the device per second.wkB/s    The number of kilobytes written to the device per second.rMB/s    The number of megabytes read from the device per second.wMB/s    The number of megabytes written to the device per second.avgrq-sz    The average size (in sectors) of the requests that were issued to the device.avgqu-sz    The average queue length of the requests that were issued to the device.await    The  average  time  (in  milliseconds)  for  I/O  requests  issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.svctm    The average service time (in milliseconds) for I/O requests that were issued to the  device.  Warning! Do not trust this field any more. This field will be removed in a future sysstat version.%util    Percentage  of  CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%.

Network Filesystem report

rBlk_nor/s    Indicate the number of blocks read by applications via the read(2) system call interface.  A  block has a size of 512 bytes.wBlk_nor/s    Indicate the number of blocks written by applications via the write(2) system call interface.rBlk_dir/s    Indicate the number of blocks read from files opened with the O_DIRECT flag.wBlk_dir/s    Indicate the number of blocks written to files opened with the O_DIRECT flag.rBlk_svr/s    Indicate the number of blocks read from the server by the NFS client via an NFS READ request.wBlk_svr/s    Indicate the number of blocks written to the server by the NFS client via an NFS WRITE request.rkB_nor/s    Indicate the number of kilobytes read by applications via the read(2) system call interface.wkB_nor/s    Indicate the number of kilobytes written by applications via the write(2) system call interface.rkB_dir/s    Indicate the number of kilobytes read from files opened with the O_DIRECT flag.wkB_dir/s    Indicate the number of kilobytes written to files opened with the O_DIRECT flag.rkB_svr/s    Indicate the number of kilobytes read from the server by the NFS client via an NFS READ request.wkB_svr/s    Indicate  the number of kilobytes written to the server by the NFS client via an NFS WRITE request.rMB_nor/s    Indicate the number of megabytes read by applications via the read(2) system call interface.wMB_nor/s    Indicate the number of megabytes written by applications via the write(2) system call interface.rMB_dir/s    Indicate the number of megabytes read from files opened with the O_DIRECT flag.wMB_dir/s    Indicate the number of megabytes written to files opened with the O_DIRECT flag.rMB_svr/s    Indicate the number of megabytes read from the server by the NFS client via an NFS READ request.wMB_svr/s    Indicate the number of megabytes written to the server by the NFS client via an NFS WRITE  request.ops/s    Indicate the number of operations that were issued to the filesystem per second.rops/s    Indicate the number of ’read’ operations that were issued to the filesystem per second.wops/s    Indicate the number of ’write’ operations that were issued to the filesystem per second.

　Command Parameters

-C: Display CPU usage-d: Display disk usage-k in KB-m in MB-N: Display disk array (LVM) information-n display NFS usage-p [disk] display disk and partition information-t display terminal and CPU information-x Display details-V display version information

Iii. Usage Device Utilization Report

[root@localhost ~]# iostat -d -k 2 4Linux 2.6.32-431.11.2.el6.x86_64 (localhost)        01/08/2016      _x86_64_        (4 CPU)Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtnxvda            114.22        13.80       713.21  372234513 19241775188xvdb             19.26        21.24       144.25  573067009 3891812336Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtnxvda              0.00         0.00         0.00          0          0xvdb              4.50         0.00        18.00          0         36Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtnxvda              0.00         0.00         0.00          0          0xvdb             10.50         0.00        42.00          0         84Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtnxvda              0.00         0.00         0.00          0          0xvdb             20.50         0.00        86.00          0        172

Parameter description

• Tps: the number of times the device transmits per second. Multiple logical requests can be combined into a single I/O Request device. Transmission has an uncertain size.

• KB_read/s: the amount of data read from the device per second

• KB_wrtn/s: data volume written to the device per second

• KB_read: total data volume read

• KB_wrtn: total data volume written

[root@localhost ~]# iostat -d -x -k 1 10Linux 2.6.32-431.11.2.el6.x86_64 (localhost)        01/08/2016      _x86_64_        (4 CPU)Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %utilxvda              0.02     0.69    0.56  113.65    13.80   713.21    12.73     0.00    0.03   0.48   5.53xvdb              0.02    17.88    1.07   18.19    21.24   144.27    17.19     0.13    6.71   1.69   3.25Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %utilxvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00xvdb              0.00     0.00    0.00    5.00     0.00    20.00     8.00     0.11   22.00   4.40   2.20Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %utilxvda              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00xvdb              0.00     0.00    0.00   16.00     0.00    64.00     8.00     0.80   50.19   6.81  10.90

Total IO (io)/s = r/s (read) + w/s (write)

Average wait time = single I/O service time * (1 + 2 +... + Requests-1)/Total requests

Parameter description

• Rrqm/s: the number of merge read operations per second. That is, delta (rmerge)/s.

• Wrqm/s: Number of write operations performed on merge per second. That is, delta (wmerge)/s

• R/s: The number of read I/O devices completed per second. That is, delta (rio)/s.

• W/s: the number of write I/O devices completed per second. That is, delta (wio)/s

• Rsec/s: Number of read sectors per second. That is, delta (rsect)/s

• Wsec/s: Number of write sectors per second. That is, delta (wsect)/s

• RkB/s: the number of K Bytes read per second. It is half of rsect/s, because the size of each sector is 512 bytes. (calculated)

• WkB/s: the number of K bytes written per second. Half of wsect/s (to be calculated)

• Avgrq-sz: average data size (slice) of each device I/O operation. delta (rsect + wsect)/delta (rio + wio)

• Avgqu-sz: Average I/O queue length, that is, delta (aveq)/s/1000 (because aveq is measured in milliseconds ).

• Await: average wait time for each device I/O operation (MS). That is, delta (ruse + wuse)/delta (rio + wio)

• Svctm: Average service time for each device I/O operation (MS). That is, delta (use)/delta (rio + wio)

• % Util: the percentage of time in one second for I/O operations, or the number of I/O queues in one second is not empty. that is, delta (use)/s/1000 (because the Unit of use is milliseconds)

If % util is close to 100%, it indicates that too many I/O requests are generated and the I/O system is fully loaded. This disk may have a bottleneck.
If the idle is less than 70% I/O, the load will be high. Generally, wait provides a high read speed.

Avgqu-sz indicates the data size of each operation. If the number of operations is large but the data size is small, the IO size will be small. if the data size is large, the IO data will be high, through avgqu-sz × (r/s or w/s) = rsec/s or wsec/s

CPU Utilization Report

[root@localhost ~]# iostat -c 1 10Linux 2.6.32-431.11.2.el6.x86_64 (localhost)        01/08/2016      _x86_64_        (4 CPU)avg-cpu:  %user   %nice %system %iowait  %steal   %idle           5.09    0.00    2.08    4.28    0.00   88.55avg-cpu:  %user   %nice %system %iowait  %steal   %idle           1.75    0.00    0.50    0.00    0.00   97.75avg-cpu:  %user   %nice %system %iowait  %steal   %idle           1.50    0.00    0.50    0.00    0.00   98.00avg-cpu:  %user   %nice %system %iowait  %steal   %idle           0.25    0.00    0.25    0.00    0.00   99.50

Parameter description

• % User: Percentage of CPU time in user mode

• % Nice: Percentage of CPU time in user mode with NICE Value

• % System: Percentage of CPU time in system Mode

• % Iowait: Percentage of CPU waiting for input/output completion time

• % Steal: Percentage of unconscious waiting time of the virtual CPU when the hypervisor maintains another virtual processor

• % Idle: Percentage of idle CPU time

If % iowait is too high, it indicates that the hard disk has an I/O bottleneck and % idle is too high, indicating that the CPU is idle.

If the value of % idle is high but the system response is slow, it may be that the CPU is waiting for memory allocation. In this case, the memory capacity should be increased. If the value of % idle is lower than 10, the CPU processing capability of the system is relatively low, indicating that the most necessary resource in the system is the CPU.

Common usage

Iostat-d-k 1 10 # view TPS and throughput information iostat-d-x-k 1 10 # view device usage (% util), response time (await) iostat-c 1 10 # view cpu status

 

References

Http://www.cnblogs.com/peida/archive/2012/12/28/2837345.html

Http://www.mjmwired.net/kernel/Documentation/iostats.txt

Http://www.orczhou.com/index.php/2010/03/iostat-detail/

Http://www.php-oa.com/2009/02/03/iostat.html