Linux SAR Command usage instructions

SAR is a very powerful performance analysis command, through the SAR command can fully obtain the system's CPU, running queue, disk I/O, swap area, memory, CPU interrupt, network and other performance data.
Common formats for the SAR command line:
SAR [Options] [-A] [-o file] t [n]
On the command line, the N and t two parameters are combined to define the sampling interval and the number of times, T is the sampling interval, is a required parameter, n is the number of samples, is optional, the default value is 1,-o file means that the command results in a binary format in the file, where file is not the keyword, is the file name. Options is a command-line option, there are a number of choices for the SAR command, and only the common options are listed below:
???-A: The sum of all reports.
???-U:CPU utilization
???-V: Process, node, file, and lock table state.
???-P: This is like specifying CPU usage information in the current system.
???-D: Hard disk usage reports.
???-r: Displays the usage of system memory.
???-N: Displays network operational status. Parameters can be followed with DEV, Edev, SOCK, and full. DEV Displays the network interface information, Edev displays the statistics of the network error, SOCK displays the socket information, full displays the first three parameters so information.


? -Q: Displays the size of the running queue, which is the same as the average load at the time of the system
? -B: Memory Paging situation
?? -r: Displays the activity of the process over the sample time.
? -G: Serial I/O condition.
? -B: Buffer usage.
? -A: File read and write.
? -C: System call condition.
? -R: The activity of the process.
? -Y: Terminal equipment activity situation.
? -W: System Exchange activity.

1, SAR monitoring CPU

Linux?2.6.39-400.17.1.el6uek.x86_64? (backupserver) October 29, 2006? _x86_64_ (24? CPU) 09:59 02 sec????? Cpu????? %user????? %nice??? %system??? %iowait???? %steal????? %idle09 59 minutes, 05 seconds????? All?????? 0.72?????? 0.00?????? 0.10?????? 0.99?????? 0.00????? 98.2009 hours, 59 minutes, 08 seconds????? All?????? 0.66?????? 0.00?????? 0.08?????? 0.58?????? 0.00????? 98.6709 hours, 59 minutes, 11 seconds????? All?????? 0.47?????? 0.00?????? 0.07?????? 0.28?????? 0.00????? 99.1809 hours, 59 minutes, 14 seconds????? All?????? 0.66?????? 0.00?????? 0.11?????? 0.36?????? 0.00????? 98.8609 hours, 59 minutes, 17 seconds????? All?????? 0.64?????? 0.00?????? 0.12?????? 0.32?????? 0.00????? 98.92 Average time:????? All?????? 0.63?????? 0.00?????? 0.10?????? 0.51?????? 0.00????? 98.77[[email protected]?~]#

??%usr: Percentage of CPU time consumed by the user process.
??%nice: The percentage of CPU time that is consumed by running a normal process.
??%system: The percentage of CPU time that the system process consumes.
??? %iowait:i/o the percentage of CPU time that the wait is taken.
??%steal: In a memory-intensive environment, Pagein enforces steal operations on different pages.
??%idle:cpu the time percentage of the idle state.
in all the display, we should pay attention to%IOW and%idle, the value of%iow is too high, indicating that there is an I/O bottleneck on the hard disk, the%idle value is high, the CPU is idle, if the%idle value is high but the system response is slow, it is possible that the CPU waits to allocate memory, this should increase the memory capacity. If the%idle value continues below 10, the system's CPU processing power is relatively low, indicating that the most resource to be addressed in the system is the CPU.

2, SAR memory and swap space monitoring

[[email protected]?~]#?sar?-r?5?3linux?2.6.39-400.17.1.el6uek.x86_ 64? (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:00 57 seconds? kbmemfree?kbmemused?? %memused?kbbuffers?? Kbcached?? Kbcommit??? %commit10 01 minutes, 02 seconds???? 357864?? 65689092????? 99.46???? 440880?? 59862196??? 5357776?????? 6.3910 hours, 01 minutes, 07 seconds???? 356620?? 65690336????? 99.46???? 440880?? 59862684??? 5359028?????? 6.3910 hours, 01 minutes, 12 seconds???? 356608?? 65690348????? 99.46???? 440880?? 59862692??? 5359000?????? 6.39 Average time:???? 357031?? 65689925????? 99.46???? 440880?? 59862524??? 5358601?????? 6.39[[email protected]?~]#?free?-m????????????? Total??????? Used??????? Free????? Shared???? Buffers????? Cachedmem:????????? 64498?????? 64150???????? 348?????????? 0???????? 430?????? 58459-/+?buffers/cache:??????? 5260?????? 59238Swap:???????? 17407????????? ?????? 17347[[email protected]?~]# 

? ? Kbmemfree: This value is basically the same as the free value in the command, so it does not include buffer and cache space.
? ? Kbmemused: This value is basically the same as the used value in the free command, so it includes buffer and cache space.
? ? %memused: This value is a percentage of kbmemused and total memory (excluding swap).
? ? Kbbuffers and kbcached: These two values are the buffer and cache in the free command.
? ? Kbcommit: Ensure that the current system requires memory (RAM+SWAP) in order to ensure that it does not overflow.
? ? %commit: This value is a percentage of kbcommit and total memory (including swap).

3, SAR memory paging monitoring

[[email protected]?~]#?sar?-b?5?3linux?2.6.39-400.17.1.el6uek.x86_64?] (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:03 23 seconds?? Pgpgin/s?pgpgout/s??? Fault/s?? Majflt/s?? PGFREE/S?PGSCANK/S?PGSCAND/S?PGSTEAL/S???? %vmeff10 03 minutes, 28 seconds?????? 0.00??? 1750.58????? 61.00?????? 0.00???? 145.95?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 03 minutes, 33 seconds?????? 0.00??? 1273.96????? 46.79?????? 0.00????? 56.23?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 03 minutes, 38 seconds?????? 0.00??? 2804.50????? 81.80?????? 0.00???? 177.67?????? 0.00?????? 0.00?????? 0.00?????? 0.00 Average Time:?????? 0.00??? 1946.11????? 63.25?????? 0.00???? 126.57?????? 0.00?????? 0.00?????? 0.00?????? 0.00[[email protected]?~]#

? ? PGPGIN/S: Represents the number of bytes per second that are displaced from disk or SWAP to memory (KB).
? ? PGPGOUT/S: Represents the number of bytes per second that are displaced from memory to disk or SWAP (KB).
? ? FAULT/S: The number of pages per second that the system generates, that is, the sum of the primary and secondary page faults (major + minor).
? ? MAJFLT/S: The number of main pages generated per second.

4. Sar I/O and transmission rate monitoring

[[email protected]?~]#?sar?-b?5?3linux?2.6.39-400.17.1.el6uek.x86_64?] (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:06 16 sec??????? Tps?????? Rtps?????? Wtps??? Bread/s??? Bwrtn/s10 06 minutes, 21 seconds???? 112.27?????? 0.74???? 111.52????? 53.53?? 10257.2510 hours, 06 minutes, 26 seconds????? 77.46?????? 0.00????? 77.46?????? 0.00??? 7848.4810 hours, 06 minutes, 31 seconds????? 93.51?????? 0.00????? 93.51?????? 0.00??? 6970.99 Average time:????? 94.53?????? 0.25????? 94.28????? 18.11??? 8374.34[[email protected]?~]#

?? TPS: The total number of I/O transfers per second for physical devices.
RTPS: The total amount of data read from a physical device per second.
Wtps: The total amount of data written to the physical device per second.
BREAD/S: The amount of data that is read from the physical device per second, in blocks/s.
BWRTN/S: The amount of data written to the physical device per second, in block/S.

5, SAR process queue length and average load status monitoring

[[email protected]?~]#?sar?-q?5?3linux?2.6.39-400.17.1.el6uek.x86_64?] (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:10 59 sec??? Runq-sz?? Plist-sz??? Ldavg-1??? Ldavg-5?? ldavg-1510 11 minutes, 04 seconds????????? 0??????? 331?????? 0.42?????? 0.40?????? 0.3910 hours, 11 minutes, 09 seconds????????? 0??????? 331?????? 0.63?????? 0.44?????? 0.4010 hours, 11 minutes, 14 seconds????????? 1??????? 331?????? 0.58?????? 0.43?????? 0.40 Average time:????????? 0??????? 331?????? 0.54?????? 0.42?????? 0.40[[email protected]?~]#

? ? Runq-sz: The length of the run queue (number of processes waiting to run).
? ? Plist-sz: The number of processes (processes) and threads (threads) in the process list.
? ? Ldavg-1: The average system load for the last 1 minutes (systems load average).
? ? LDAVG-5: Average system load over the last 5 minutes.
? ? LDAVG-15: Average system load over the last 15 minutes.

6. SAR system Exchange activity information monitoring

[[email protected]?~]#?sar?-w?3?5linux?2.6.39-400.17.1.el6uek.x86_64?] (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:13 07 seconds?? Pswpin/s?pswpout/s10 13 minutes, 10 seconds?????? 0.00?????? 0.0010 hours, 13 minutes, 13 seconds?????? 0.00?????? 0.0010 hours, 13 minutes, 16 seconds?????? 0.00?????? 0.0010 hours, 13 minutes, 19 seconds?????? 0.00?????? 0.0010 hours, 13 minutes, 22 seconds?????? 0.00?????? 0.00 Average Time:?????? 0.00?????? 0.00[[email protected]?~]#

PSWPIN/S: The number of swap pages (Swap page) that the system swapped in per second.
PSWPOUT/S: The number of swap pages (Swap page) that the system swapped out per second.

7, the use of SAR equipment monitoring

[[email protected]?~]#?sar?-d?5?3linux?2.6.39-400.17.1.el6uek.x86_64? (backupserver) October 29, 2006? _x86_64_ (24? CPU) 10:14 15 sec??????? DEV??????? Tps?? Rd_sec/s?? Wr_sec/s?? Avgrq-sz?? Avgqu-sz????? await????? SVCTM????? %util10 14 minutes, 20 seconds???? dev8-0????? 23.67?????? 0.00??? 3457.58???? 146.05?????? 0.19?????? 8.18?????? 4.69????? 11.1010 hours, 14 minutes, 20 seconds?? dev252-0?????? 0.57?????? 0.00?????? 4.55?????? 8.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 20 seconds?? dev252-1?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 20 seconds?? Dev252-2?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 20 seconds?? Dev252-3?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 20 seconds?? Dev252-4????? 58.71?????? 0.00??? 3453.03????? 58.81?????? 0.44?????? 7.56?????? 1.89????? 11.1010 hours, 14 minutes, 20 seconds??????? DEV??????? Tps?? Rd_sec/s?? Wr_sec/s?? Avgrq-sz?? Avgqu-sz????? await????? SVCTM????? %util10 14 minutes, 25 seconds???? dev8-0????? 31.06?????? 0.00??? 4545.09???? 146.32?????? 0.22?????? 6.94?????? 2.93?????? 9.1010 hours, 14 minutes, 25 seconds?? dev252-0?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 25 seconds?? dev252-1?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 25 seconds?? Dev252-2?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 25 seconds?? Dev252-3?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 25 seconds?? Dev252-4????? 69.74?????? 0.00??? 4545.09????? 65.17?????? 0.33?????? 4.75?????? 1.30?????? 9.1010 hours, 14 minutes, 25 seconds??????? DEV??????? Tps?? Rd_sec/s?? Wr_sec/s?? Avgrq-sz?? Avgqu-sz????? await????? SVCTM????? %util10 14 minutes, 30 seconds???? dev8-0????? 38.11?????? 0.00??? 5860.90???? 153.77?????? 0.64????? 16.91?????? 3.99????? 15.2310 hours, 14 minutes, 30 seconds?? dev252-0?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 30 seconds?? dev252-1?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 30 seconds?? Dev252-2?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.001014 minutes, 30 seconds?? Dev252-3?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.0010 hours, 14 minutes, 30 seconds?? Dev252-4????? 87.82?????? 0.00??? 5860.90????? 66.74?????? 0.91????? 10.41?????? 1.74????? 15.25 Average Time:??????? DEV??????? Tps?? Rd_sec/s?? Wr_sec/s?? Avgrq-sz?? Avgqu-sz????? await????? SVCTM????? %util Average time:???? dev8-0????? 30.86?????? 0.00??? 4607.29???? 149.30?????? 0.35????? 11.35?????? 3.83????? 11.82 average time:?? dev252-0?????? 0.20?????? 0.00?????? 1.56?????? 8.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00 Average time:?? dev252-1?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00 Average time:?? Dev252-2?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00 Average time:?? Dev252-3?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00?????? 0.00 Average time:?? Dev252-4????? 71.94?????? 0.00??? 4605.73????? 64.02?????? 0.56?????? 7.83?????? 1.64????? 11.82[[email protected]?~]#

? ?? TPS: Number of times per second from physical disk I/O. Multiple logical requests are merged into one I/O disk request, and the size of one transfer is indeterminate.
? ? RD_SEC/S: Number of Read sectors per second.
? ? WR_SEC/S: Number of Write sectors per second.
? ? Avgrq-sz: The average data size (sector) per device I/O operation.
? ? Avgqu-sz: The average length of the disk request queue.
? ? Await: The average elapsed time of each request, including the request queue wait time, in milliseconds (1 seconds =1000 milliseconds), from the request disk operation to the completion of the system processing.
? ? SVCTM: The average time that the system processes each request, excluding the time consumed in the request queue.
? ? %UTIL:I/O requests account for the percentage of the CPU, the higher the ratio, the more saturated the description.
? ? 1. When the value of Avgqu-sz is low, the equipment utilization is higher.
? ? 2. When the value of%util is close to 1%, it indicates that the device bandwidth is already full.

Summarize

To judge a system bottleneck, sometimes it takes several SAR command options to combine.
?? suspected CPU bottleneck, can be used sar-u and sar-q, etc. to view
?? suspected memory bottlenecks, available sar-b, Sar-r, and sar-w to view
?? suspected I/O bottlenecks, available sar-b, sar-u, and sar-d to view

Linux SAR Command usage instructions