Mpstat and SAR Addendum

Sysstat is a tool set, including SAR, pidstat, iostat, mpstat, SADF, and SADC. Among them, SAR is the most powerful and the most suitable tool for testing.

Installation Steps

You can download it at http://pagesperso-orange.fr/sebastien.godard/download.html.

1 Installation

Tar zxvf xxx.tar.gz

./Configure

Make

Make install

2. mpstat first

Mpstat is short for Multiprocessor statistics and is a real-time system monitoring tool. Its report and CPU statistics are stored in the/proc/STAT file. In a multi-CPUs system, it can not only view the average status information of all CPUs, but also view information about specific CPUs. The following describes only the CPU-related mpstat parameters. The mpstat syntax is as follows: mpstat [-P {| all}] [internal [count]

The parameters are described as follows:

Parameter description

-P {| all} indicates the CPU to be monitored. The CPU value ranges from 0 to 1.

For example, mpstat-P all 1

Interval between two adjacent internal sampling

Count: the number of samples. Count can only be used with delay.

If no parameter exists, mpstat displays the average value of all information after the system is started. The average information of the first line since the system was started when interval exists. From the second line, the output is the average information of the previous interval period. CPU-related output is described as follows:

Get data from/proc/STAT

CPU processor ID

User-state CPU time (%) during the internal period, excluding nice value for negative process usr/total * 100

Nice in the internal period, nice value is the CPU time of the negative process (%) Nice/total * 100

System in the internal time period, the core time (%) system/total * 100

Hard Disk I/O wait time (%) iowait/total * 100 during the internal Period

IRQ in the internal period, Soft Interrupt time (%) IRQ/total * 100

Soft Interrupt time (%) softirq/total * 100 during internal Period

Idle in the internal period, the idle time (%) idle/total * 100 (excluding the idle time (%) of CPU waiting for disk I/O operations for any reason)

Intr/s: the number of CPU interruptions received per second during the internal period. intr/total * 100

Total CPU time = total_cur = user + system + nice + idle + iowait + IRQ + softirq

By the way

CAT/proc/STAT

"Ctxt" indicates the number of times the CPU context has been exchanged since the system was started.

"Btime" indicates the time (in seconds) that has elapsed since the system was started.

The number of tasks created since the system started "processes (total_forks.

"Procs_running": number of tasks in the current queue.

"Procs_blocked": Number of blocked tasks.

3 SAR sampling

SAR sampling prior to http://jackyrong.iteye.com/blog/238663

As mentioned above, here is a supplement:

SAR [Options] [-A] [-o file] T [N]

In the command line, the N and T parameters are combined to define the sampling interval and number of times. t indicates the sampling interval, which must be

N is the number of samples and is optional. The default value is 1.-o file indicates that the command result is in binary format.

Stored in a file. The file is not a keyword but a file name. Options is the command line option, sar command

There are many options, and only common options are listed below:

-A: total of all reports.

-U: CPU usage

-V: process, I node, file, and lock table status.

-D: hard disk usage report.

-R: memory and swap space usage statistics.

-G: serial port I/O.

-B: Buffer usage.

-A: file read/write status.

-C: System Call status.

-Q: Report queue length and average system load

-R: Process activity.

-Y: terminal device activity.

-W: system exchange activity.

-X {pid | self | all}: reports statistics of the specified process ID. The self keyword is the statistics of the SAR process, and the All keyword is the statistics of all system processes.

Analysis of CPU utilization using SAR # Sar-U 2 10

Linux 2.6.18-53. el5pae (localhost. localdomain) 03/28/200907: 40: 17 Pm CPU % USER % nice % System % iowait % steal % idle

% USER: Percentage of CPU time in user mode. % Nice: Percentage of CPU time in user mode with nice value.

% SYSTEM: Percentage of CPU time in system mode.

% Iowait: Percentage of CPU waiting for input/output completion time. % Steal: Percentage of unconscious waiting time of the virtual CPU when the hypervisor maintains another virtual processor.

% Idle: Percentage of idle CPU time. In all the displays, we should pay attention to % iowait and % idle. The value of % iowait is too high, indicating that the hard disk has an I/O bottleneck, and the value of % idle is high, indicating that the CPU is idle, if the value of % idle is high but the system response is slow, the CPU may be waiting for memory allocation. In this case, the memory capacity should be increased. If the value of % idle is lower than 10, the CPU processing capability of the system is relatively low, indicating that the most important resource to be solved in the system is the CPU. Analyze the queue length of running processes using SAR: # Sar-Q 2 10

Linux 2.6.18-53. el5pae (localhost. localdomain) 03/28/200907: 58: 14 PM runq-SZ plist-SZ ldavg-1 ldavg-5 ldavg-15

0.49runq-SZ: The process running queue to be run.

Plist-SZ Number of processes and threads in the process queue

4. Be careful not to be superstitious!

(From http://developers.sun.com.cn/blog/yutoujava/entry/14)

Performance tests on the t2000 server often find that sometimes it is strange to use mpstat or vmstat to observe the same system, that is, when the CPU usage is low, the performance is better, but the response time is poor when the CPU is fully utilized. The more CPU usage, the more commands are executed and the more jobs are performed. Why is it slow?

The t2000 server has a single CPU, but the CPU has eight cores, each of which has four hardware threads, and 32 virtual CPUs are displayed in the Solaris operating system. In fact, each core has an independent execution unit (or pipeline). Therefore, t2000 can generally be equivalent to an 8-CPU machine. Each core has four hardware threads, mainly preparing executable instruction sets for each core. The utilization rate of hardware threads and the utilization rate of cores are two different concepts. Mpstat and vmstat are mainly used in a single hardware thread environment. In a multi-threaded CPU, the displayed results are different.

Generally, if a CPU is not in the idle status, mpstat or vmstat considers the CPU as busy. When the CPU executes commands, it will stop in some situations and no longer execute any commands. However, in a non-multithreading CPU, if the current CPU cannot execute other commands for some reason, it will also be treated as busy. This situation often occurs. For example, some large commands (load) may span several command cycles. During this period, the CPU cannot execute other commands temporarily, so it is also counted as busy. However, in a multi-threaded CPU, the CPU can switch to execute commands in other threads during any pause. Therefore, the traditional mpstat or vmstat cannot reflect this situation.

So why does mpstat or vmstat indicate that the usage is low, but the system performance is better. This is because, for the four threads in one core, although the CPU usage of each thread is not high (for example, each 25%), the execution unit of this core is fully utilized.

Sometimes, the CPU usage of each thread is very high, and the CPU (CORE) is not necessarily working. For example, each thread of this core is executing the load command. At these times, the CPU of the four threads is 100% busy, but the CPU does not execute anything within a period of time.

In a multi-threaded CPU environment, other operating system commands are required to truly observe CPU utilization, such as cpustat:

Cpustat-n-C pic0 = l2_dmiss_ld, pic1 = pai_cnt-C pic0 = l2_dmiss_ld, pic1 = pai_cnt, nouser, sys 1

Alternatively, go to http://cooltools.sunsource.net/corestat/to download the corestattool package.