Linux Performance Monitoring: Introduction

Looked at such a tutorial, read a manual, according to the requirements to change a set, system settings, kernel parameters that the idea of a system optimization is silly very naïve: the system optimization is a complex, cumbersome, long-term work, optimization needs to be monitored, collected, tested, evaluated, optimized after testing, collection, evaluation, monitoring, And is a long-term and continuous process, not to say now optimization, testing, and then once and for all, or not to say that the book optimization is suitable for the current operating system, different systems, different hardware, different application optimization of the focus of different, optimized methods are different, optimized parameters are also different. Performance monitoring is an important part of the system optimization process, if there is no monitoring, not clear where the performance bottlenecks, optimize what, how to optimize it? Therefore, to find the performance bottleneck is the goal of performance monitoring, but also the key to system optimization. The system is composed of several subsystems, usually modifying one subsystem may affect another subsystem, even cause the whole system to be unstable and crash. So optimization, monitoring, testing is usually linked together, and is a cyclical and long-term process, usually monitored subsystems have the following:

• Cpu
• Memory
• Io
• Network

These subsystems depend on each other, understand the characteristics of these subsystems, monitoring the performance parameters of these subsystems and timely detection of possible bottlenecks for system optimization is very helpful.

Application Type

Different system uses are also different, to find performance bottlenecks need to know what the system is running the application, some characteristics, such as Web server requirements for the system and file server is not the same, so it is important to distinguish between different types of application of the system, usually the application can be divided into two types:

• IO-related, IO-related applications are often used to process large amounts of data, require a lot of memory and storage, frequent IO operations to read and write data, and less CPU requirements, most of the time the CPU is waiting for the hard disk, such as the database server, file server, etc.
• CPU-related, CPU-related applications need to use a large number of CPUs, such as high-concurrency Web/mail servers, image/video processing, scientific computing, etc. can be regarded as CPU-related applications.

Take a look at the actual example, the 1th is the feature that the file server displays when copying a large file:

$ vmstat 1procs-----------memory-------------Swap-------io------System-------CPU-----r  B   swpd   free   buff  cache   si   so    bi    bo   in   CS US sy ID WA St 0  4    140 1962724 335516 4852308< C13/>0    0   388 65024 1442  563  0  2  0 0 4 $    1961816 335516 4853868  0< C23/>0   768 65536 1434  522  0  1  0 0 4 $    1960788 335516 4855300  0    0< c33/>768 48640 1412  573  0  1  0 0 4 $    1958528 335516 4857280  0    0  1024x768 65536 1415  521  0  1  0 0  5    1957488 335516 4858884  0    0   768 81412 1504  609  0  2  0

The 2nd is the characteristics of the CPU when it makes a lot of calculations:

$ vmstat 1procs-----------memory-------------Swap-------io------System-------CPU-----r  B   swpd   free   buff  cache   si   so    bi    bo   in   CS US sy ID WA St 4  0    140 3625096 334256 3266584< C13/>0    0     0    1054  470 0  0  0 0 4  0    3625220 334264 3266576  0    0     0    1037  448 0  0  0 0 4  0    3624468 334264 3266580  0    0     0   148 1160  632 0  0  0 0 4  0    3624468 334264 3266580  0    0     0     0 1078  527 0  0  0 0 4 0    3624712 334264 3266580  0    0     0    1053  501 0  0  0  0

The most obvious difference between the above two examples is the ID column, which represents the CPU's idle rate, the ID of the copy file is maintained at about 50%, and the CPU is calculated with a base of 0.

Bottom line

How do we know if the system performance is good or bad? This requires a baseline, if the performance monitoring of the statistical data across the line, we can say that the system performance is poor, if the data can be kept online we say the performance is good. Building such a bottom line requires knowing some theory, additional load testing, and years of experience with system administrators. If you do not have many years of experience, there is a simple way to draw the bottom line is to set the bottom line in their expectations of the system. Expect the system to have a performance, this is a bottom line, if not meet this requirement is poor performance. For example, Vpsee last month had a RAID0 test, the expected test results should be RAID0 IO performance than a single drive significantly improved, the bottom line is that RAID0 io is at least better than a single hard drive (how much less important, the bottom line is at least), test results found RAID0 performance is not as hard as a single Disk, indicating poor performance, this time need to ask why, this is often a performance bottleneck, after the investigation found that the original hard disk hardware defects caused by the performance test results error.

Monitoring Tools

We only need simple tools to monitor the performance of Linux, the following are the tools commonly used by Vpsee:

tool	Brief introduction
Top	View process activity status and some system conditions
vmstat	View system Status, hardware and system information, etc.
iostat	View CPU load, hard disk status
SAR	Synthesis tool, viewing system health
mpstat	viewing multiprocessor status
netstat	view network status
IPT RAF	Real-time network condition monitoring
tcpdump	Crawl network packets, detailed analysis
tcptrace	Packet Analysis tool
netperf	Network bandwidth tool
dstat	integrated Tools , combined with Vmstat, Iostat, Ifstat, Netstat, and more.

A fully running Linux system consists of many subsystems (introduction, Cpu,memory,io,network, ...). ), monitoring and evaluating these subsystems is part of the performance monitoring. We often need macroscopic view of the whole system state, also need microscopic look at the operation of each subsystem.

Fortunately, we don't have to reinvent the wheel to monitor these subsystems with the tools available, and the time-tested, simple and elegant gadgets that grow with Unix are an integral part of our daily unix/linux work.

The following picture summarizes the various subsystems of Linux and the tools needed to monitor them, and if you are interested in Linux system Management (sysadmin & DevOps) and want to get started, you can begin to understand and familiarize yourself with the tools from this diagram. For the skilled Linux cock wire, you should be able to answer this picture with ease. (Image from: Linux performance analysis and Tools, slide is also very exciting, recommend reading against.) ）

Linux Performance Monitoring: Introduction