Performance analysis of Linux system performance tuning

The purpose of 1.Linux performance analysis
1) Identify system performance bottlenecks (including hardware bottlenecks and software bottlenecks);
2) provide a performance-optimized solution (upgrade the hardware?) Improve system system structure? ）；
3) to achieve reasonable hardware and software configuration;
4) Maximize the balance of system resource usage. (in general, the system runs well when the resources reach a balance, the transition of any one of the resources will cause the balance system damage, resulting in a very high system load or slow response.) For example, CPU transition will cause a large number of processes to wait for CPU resources, the system response is slow, waiting will cause the number of processes, the increase of the process will cause memory usage increase, memory exhaustion will cause virtual memory usage, and use virtual memory will cause disk IO increase and CPU overhead.

2. Factors that affect performance
1) CPU (CPU speed and performance of a large part of the system determines the overall performance, whether using SMP)
2) memory (when physical memory is not enough to use swap memory, using swap will bring disk I0 and CPU overhead) BR data-filtered= "Filtered" >3) hard disk (storage System)
a.raid technology use (RAID0, RAID1, RAID5, raid0+1)
b. Small file read/write bottlenecks are disk addressing (TPS), and the performance bottleneck for large file reads and writes is bandwidth
c. Linux can use free memory as the cache for file system access, so the larger the system memory, the better the performance of the storage System
4) network bandwidth.

3. Steps for performance analysis
1) long-term monitoring and data acquisition of the use of resources (Nagios, cacti)
2) Use common performance analysis tools (Vmstat, top, free, iostat, etc.)
3) Accumulated experience
A. Application design flaws and database query misuse are most likely to lead to performance issues
B. Performance bottlenecks may be due to poor program/memory/disk bottlenecks, but ultimately the result is CPU exhaustion, high system load, slow response, and even transient loss of response
C. Swap memory is used when physical memory is insufficient, and swap will bring disk I0 and CPU overhead
D. Problems that can cause CPU bottlenecks: Frequent Perl,php,java programs generate dynamic web; database queries a large number of where clauses, order by/group by ordering ...
E. Possible memory bottlenecks: High concurrent user access, system processes, Java memory leaks ...
F. Possible disk IO bottleneck problems: Generate cache files, database updates frequently, or query large tables ...

4.vmstat Detailed Introduction
Vmstat is a comprehensive profiling tool that observes the system's process state, memory usage, virtual memory usage, disk IO, interrupts, context switches, CPU usage, and more. For Linux performance analysis, 100% understand the meaning of vmstat output content, and can be used flexibly, the ability to analyze the system performance is basically mastered.

Linux vmstat Command Detailed http://www.linuxidc.com/Linux/2013-03/80983.htm

Vmstat display results in Linux under the detailed http://www.linuxidc.com/Linux/2013-01/77497.htm

Linux Monitoring Tools Vmstat detailed http://www.linuxidc.com/Linux/2012-09/71295.htm

Linux vmstat Command Real-combat detailed http://www.linuxidc.com/Linux/2012-06/61771.htm

Linux vmstat Monitoring System load http://www.linuxidc.com/Linux/2012-01/51340.htm

Vmstat Command detailed--linux performance analysis http://www.linuxidc.com/Linux/2014-06/102884.htm

The following is the output of the Vmstat command:
# vmstat 1 5

The output is interpreted as follows:
1) procs
A.R column represents the number of processes running and waiting for CPU time slices, if the long-term is greater than the number of system CPUs, it indicates that CPU resources are insufficient, you can consider increasing the CPU;
The b.b column represents the number of processes waiting on a resource, such as waiting for I/O or memory exchange.
2) Memory
The A.SWPD column represents the amount of memory (in kilobytes) that is switched to the memory swap area. If the value of SWPD is not 0 or larger, and the value of Si, so long 0, then this situation generally do not have to worry about, will not affect the system performance;
The B.free column represents the amount of physical memory currently idle in kilobytes (KB);
The C.buff column represents the amount of memory buffers cache, which is generally required to read and write to the block device.
The D.cache column represents the amount of memory for page cached, which is generally used as the cached of the file system, and frequently accessed files are cached. If the cached value is large, it indicates that there are many cached files. If the bi in IO is small in this case, the file system is more efficient.
3) Swap
The A.si column indicates the amount of memory that is being transferred from the disk into the memory swap area;
The b.so column represents the amount of memory that is called into the disk, which is the memory swap area
C. In general, the values of Si and so are 0, if the value of Si and so is not 0 for a long time, it indicates that the system memory is not enough and needs to consider whether to increase system memory.
4) IO
The A.bi column represents the total amount of data read from the block device (that is, read disk, Unit kb/sec)
B.bo column indicates the total amount of data written to the block device (that is, write disk, Unit kb/sec)
The Bi+bo reference value set here is 1000, if more than 1000, and the WA value is larger, it represents the system disk IO performance bottleneck.
5) System
The a.in column represents the number of device interrupts per second observed in a time interval;
The B.cs column represents the number of context switches produced per second.
The higher these two values are, the more CPU time the kernel consumes.
6) CPU
The a.us column shows the percentage of time that the user process consumes the CPU. When the value of us is higher, it indicates that the user process consumes more CPU time, if the long-term greater than 50%, need to consider the optimizer what.
The B.sy column shows the percentage of time that the kernel process consumes the CPU. When the value of SY is higher, it indicates that the kernel consumes more CPU time, and if Us+sy exceeds 80%, it indicates that there is insufficient CPU resources.
The C.id column shows the percentage of time the CPU is in idle state;
The D.wa column represents the percentage of CPU time that the IO wait occupies. The higher the WA value, the more serious the IO waits. If the WA value exceeds 20%, the IO wait is severe.
The E.st column is generally not concerned with the percentage of time that the virtual machine occupies. (Linux 2.6.11)

This article was reproduced from: https://www.linuxidc.com/Linux/2015-05/117056.htm

Performance analysis of Linux system performance tuning