The Vmstat of Linux system tuning

Find bottlenecks

The system aspect that you are most interested in in metrics should be CPU usage and memory utilization.

It is possible that the CPU will not be sufficient to handle the workload even after full tuning, in which case the CPU upgrade is required. Similarly, if the program is running in an environment that does not provide enough memory after tuning, you must increase the memory capacity.

If a performance problem is caused by any one of several factors, you must find multiple areas to eliminate each problem. First, identify the resources that restrict the system:

• Cpu
• Memory
• Input/output (I/O)

To do this, use theVmstatCommand.Vmstatcommand to generate a concise report detailing the activities of the three regions:

> vmstat 1 10  

Output:

KTHR memory page faults CPU------- --------------------------------------------------------R b AVM fre re Pi po fr SR CY in SY CS US sy i D WA 0 0 189898 612 0 0 0 3 One 0 178 606 424 6 1 1 1 0 189898 611 0 1 0 0 0 0 4573 122 4 0 0  1 0 189898 611 0 0 0 0 0 0 115 420 102 99 0 0 0 1 0 189898 611 0 0 0 0 0 0 115 425 91 99 0 0 0 1 0 189898 611 0 0 0 0 0 0 114 428 90 99 0 0 0 1 0 189898 610 0 1 0 0 0 0 117 333 102 97 3 0 0 1 0 189898 610 0 0 0 0 0 0 114 433 91 99 1 0 0 1 0 189898 610 0 0 0 0 0 0 114 429 94 99 1 0 0 1 0 189898 610 0 0 0 0 0 0 437 94 0 0 0 1 0 189898 609 0 1 0 0 0 0 340 98 2 0 0 

The previous example shows a CPU-constrained system. This is visible as the user (US) plus the System (SY) CPU value (equal to or closer to 100). Memory-Constrained Systems Display page in (PI) and page out (PO) valid values. The disk I/O throttling system displays more than 10% I/O wait percent (WA). More details about Vmstat can be found in the AIX debug command.

The following references http://www.cnblogs.com/ggjucheng/archive/2012/01/05/2312625.html

Linux vmstat Command Real-combat detailed

The Vmstat command is the most common Linux/unix monitoring tool that can show the status of a server at a given time interval, including server CPU utilization, memory usage, virtual memory exchange, IO Read and write situations. This command is I see Linux/unix favorite commands, one is Linux/unix support, two is compared to top, I can see the whole machine CPU, memory, IO usage, rather than just see the CPU usage and memory utilization of each process (using the scene is not the same).

The use of the general Vmstat tool is done by two numeric parameters, the first parameter is the number of time intervals sampled, the unit is seconds, the second parameter is the number of samples, such as:

[Email protected]:~# vmstat2 1procs-----------Memory-------------Swap-------io-----system------CPU----r b swpd free buff cache si so Bi boinchCS US sy ID WA1  0      0 3498472 315836 3819540    0    0     0     1    2    0  0  0  -  0

2 indicates that the server state is collected every two seconds, and 1 means that it is collected only once.

In fact, in the application process, we will be monitoring for a period of time, do not want to monitor the direct end of Vmstat, such as:

[Email protected]:~# vmstat2procs-----------Memory-------------Swap-------io-----system------CPU----r b swpd free buff cache si so Bi boinchCS US sy ID WA1  0      0 3499840 315836 3819660    0    0     0     1    2    0  0  0  -  0 0  0      0 3499584 315836 3819660    0    0     0     0    the  158  0  0  -  0 0  0      0 3499708 315836 3819660    0    0     0     2    the  162  0  0  -  0 0  0      0 3499708 315836 3819660    0    0     0    Ten   Bayi  151  0  0  -  0 1  0      0 3499732 315836 3819660    0    0     0     2    the  154  0  0  -  0

This means that Vmstat collects data every 2 seconds and collects it until I have finished the program, and I have collected 5 data and I have finished the program.

Well, the command is complete, now the actual combat to explain the meaning of each parameter.

R means running the queue (that is, how many processes are really allocated to the CPU), the server I am testing is currently idle, there is no program running, when this value exceeds the number of CPUs, there will be a CPU bottleneck. This is also related to top of the load, the general load over 3 is relatively high, more than 5 is high, more than 10 is not normal, the state of the server is very dangerous. The load on top is similar to the run queue per second. If the running queue is too large, it means that your CPU is busy, which generally results in high CPU usage.

b represents the blocking process, which is not much to say, the process is blocked, you understand.

swpd Virtual memory has been used size, if greater than 0, indicates that your machine is out of physical memory, if not the cause of program memory leaks, then you should upgrade the memory or the memory-consuming task to other machines.

Free physical memory size, my machine memory total 8G, the remaining 3415M.

Buff Linux/unix system is used to store, directory inside what content, permissions, etc. of the cache, I machine about more than 300 m

the cache cache is used directly to memorize the files we open, to buffer the files, I have about 300 m of this machine (this is the smart place of Linux/unix, the spare part of the physical memory to do the file and directory cache, is to improve the performance of the program execution, When the program uses memory, buffer/cached is quickly used. )

Si reads the size of the virtual memory from disk every second, if this value is greater than 0, it means that the physical memory is not enough or the memory leaks, to find out the memory process. My machine has plenty of memory and everything is fine.

so per second the virtual memory is written to the size of the disk, if this value is greater than 0, ibid.

The number of blocks received per second by BI block devices, where the block device refers to all the disks and other block devices on the system, the default block size is 1024byte, I have no IO operation on this machine, so it has been 0, but I have been working on copying large amounts of data (2-3T) The machine has seen can reach 140000/s, disk write speed of almost 140M per second

Bi:blocks received from a block device (BLOCKS/S). -The number of blocks received per second from the block device, which is the read block device.
Bo:blocks sent to a block device (BLOCKS/S). -The number of blocks sent to the block device per second, which is the write block device. bi and Bo are generally close to 0, otherwise the IO is too frequent and needs to be adjusted.

CS per second, such as the number of context switches, such as we call the system function, the context switch, the thread of the switch, but also the process context switch, the smaller the value of the better, too big, to consider the number of threads or processes, such as in Apache and Nginx Web server , we generally do performance testing will carry out thousands of concurrent or even tens of thousands of concurrent testing, the process of selecting a Web server can be the peak of the process or the thread has been down, pressure measurement, until CS to a relatively small value, the process and the number of threads is a more appropriate value. System calls are also, each time the system function is called, our code will enter the kernel space, resulting in context switching, this is very resource-intensive, but also try to avoid frequent calls to system functions. Too many context switches means that most of your CPU is wasted in context switching, resulting in less time for the CPU to do serious work, and the CPU not being fully utilized, is undesirable.

US user CPU time, I used to do encryption and decryption very frequently on the server, you can see us approaching 100,r running queue reached 80 (the machine is doing a stress test, poor performance).

sy System CPU time, if too high, indicates a long system call time, for example, the IO operation is frequent.

ID Idle CPU time, in general, ID + US + sy = 100, generally I think ID is idle CPU usage, US is the user CPU usage, SY is the system CPU utilization.

wa waits for IO CPU time.

So how do we know if the problem is caused by CPU bottlenecks because the CPU itself is not processing enough or is the CPU running too much responsibility for the algorithm or code?

We can deal with the following scenarios:

The first step: confirm the CPU bottleneck, use Vmstat 2 5 (for 2 seconds to fetch the data, take 5 times), to see the display of the US and SY ratio, if reached or close to 100%, indicating that the CPU has a bottleneck

The second step: look at the CPU consumes the most resources of the thread/process, top can see a PID CPU consumption; such as CPU 4 core, now PID utilization is 200% that is obviously PID consumption is more

Then look at the thread under the PID, if the thread is processing fast and basically does not consume CPU, that is CPU processing power is not enough



If the threading is slow and CPU consuming, it is problematic to run complex algorithms or code logic

my program creates and executes multiple threads inside of it, how can I monitor a single thread after the program creates a thread? I want to see a single thread detail with their name (for example, cpu/memory utilization).

threads are a popular programming abstraction for parallel execution on modern operating systems. When multiple threads in a program are forked to execute multiple streams, these threads share specific resources (such as memory address space, open files) between them to minimize fork splits and avoid a large number of high-cost IPC(interprocess communication) channels. These features enable threads to become an efficient mechanism when executing concurrently.

in Linux , threads created in the program (also known as lightweight processes, LWP) have the same thread group ID as the PID of the program. Each thread then obtains its own thread ID (TID). For the Linux kernel Scheduler, threads are just the standard process of sharing a specific resource. Classic command-line tools, such as PS or top, can be used to display thread-level information, but they display process-level information by default.

here are a few ways to show the threads of a process on Linux.

Method One: PS

in the PS command, the "-t" option can turn on thread viewing. The following command lists all threads created by process number <pid> processes.

1.$ ps-t-P <pid>

The SIDcolumn represents the thread ID, and the "CMD" bar displays the thread name.

Method Two: Top

Top command to display the individual thread conditions in real time. To turn on thread viewing in the top output, call the "-H" option of the top command, which lists all Linux threads. In the top run, you can also switch the thread viewing mode to on or off by pressing the "H" key.

1.$ top-h

2. 3

to have top output a specific process <pid> and check the thread condition running within that process:

$ top-h-P <pid>

The Vmstat of Linux system tuning