System Performance Detection,

System Performance Detection,

Performance monitoring focuses on computer system resource monitoring, which is necessary for preventive maintenance of the system. You can analyze the monitoring data to understand the bottlenecks of the system, measures should be taken to adjust or update affected resources. Linux provides a variety of performance monitoring tools to help administrators complete system monitoring;

1. Performance monitoring overview:

System performance monitoring and adjustment is an important part of the daily maintenance work of Linux system administrators. To measure the performance status of a system, the system response time and system throughput can be analyzed. The response time is the time required to send a request to obtain the returned result, throughput refers to the number of transactions completed by the system within a given period of time. The larger the throughput of the system, the stronger the processing capability of the system; one of the main tasks of the Administrator in performance monitoring testing is to find out the performance bottleneck of the system and then make targeted adjustments, performance bottleneck refers to the factors that have a decisive impact on the system performance. Different application systems have different performance bottlenecks. Most of the performance bottlenecks of busy file servers are disk subsystems, the performance bottleneck of a large number of online application servers may be the CPU subsystem, while the performance bottleneck of various Internet network servers is usually the network bandwidth. The system resources to be monitored mainly include CPU and memory, disks and networks;

The Red Hat 5 graphic interface provides a performance monitoring function similar to Windows Task Manager-system monitor. Choose "manage"> "system monitor" from the main menu to open the system monitor, view process, CPU, memory, network and file system information in real time;

　　　　　　　　　　

　　　　　　　　　　

Although the system monitor is very convenient, it has limited functions. to further analyze the system performance, you must use some performance monitoring tools, such as vmstat, mpstat, iostat, sar, and top, mpstat provides CPU-related data. sar is used to collect, report, and store system activity information. iostat provides CPU usage and Hard Disk Throughput data, statistics on the overall CPU activity status; top is an excellent interactive comprehensive monitoring tool; iostat and sar commands are provided by the sysstat software package and are not installed by default, install the sysstat package sysstat-7.0.0-3.el5.i386.rpm in the third disc of Red Hat 5;

2. CPU performance monitoring:

The CPU determines the computing capability of the system. All Program commands in the system are processed by the CPU. Because Linux itself is a multi-user and multi-task operating system, therefore, the CPU simultaneously processes programs with different priority levels. If too many programs are executed at the same time, the CPU may cause system performance bottlenecks. Regarding the overall performance of the CPU, you can use the sar command to view details. The basic usage of the sar command is as follows:

Sar [Option] [sampling interval] [sampling times]

The sampling interval is measured in seconds. The administrator can collect performance data for a certain period of time based on the needs to understand the system performance, to evaluate the system performance more accurately, we should analyze the performance data for a period of time rather than just a specific time point, such as collecting CPU performance data every 5 seconds, collected three times in total, run the following command:

　　　　　　　　　　

The preceding information lists the sampling time, And the CPU usage is classified by category. The last row is the average value. % user indicates the CPU usage of the user process; % nice indicates the CPU usage of the nice operation (Special process) of the user process; % system indicates the CPU usage of the system process; % iowait indicates the CPU usage consumed by waiting for disk I/O; % steal indicates the CPU usage of the virtual device; % idel indicates the percentage of CPU idle time; the sar command shows the total CPU performance. For a multi-processor system or multi-core processor, you can use the mpstat command to view the status of each CPU separately. The syntax format is as follows:

Mpstat [-p cpu number | ALL] [sampling interval] [sampling times]

Use Option-P to specify the CPU to be viewed. The CPU number starts from 0. For example, view the first CPU:

　　　　　　　　　　　

The command provides three more types of information than sar. The % irq list indicates the CPU usage of hard interrupt, And the % soft column indicates the CPU usage of soft Interrupt, the intr/s list shows the number of interruptions processed per second (times/s );

3. memory performance monitoring:

The computer memory has a certain capacity. When the amount of memory required exceeds the physical memory capacity, the system will use the paging technology and swap technology of the virtual memory, moving part or all of the program processes to the hard disk can free up space for new processes. When paging and switching are not frequent, the system is completely acceptable, when paging and switching are performed frequently, the system performance will be affected, thus forming a performance bottleneck;

1> use the free command to display various memory conditions of the system:

The free command can be used to view memory and virtual memory usage. The default unit is KB. Here is an instance:

　　　　　　　　　　　　　　　　

The Mem row displays the physical memory, the total column shows the total physical memory, and the used column shows the usage (the quantity allocated to the cache, some of which may not be actually used ), the free column indicates the available amount (memory not allocated). The shared column displays the memory shared by multiple processes, the buffers column displays the system allocated but not used buffer (the number of memories used as the buffer), and the cache column displays the cache allocated but not used by the system (the amount of memory used as the high-speed cache );

Tip: buffers and cache both occupy memory. Pay attention to their differences. buffers is the read/write cache area of Block devices, the memory that stores data to be written to the disk, and the physical memory, it performs distributed write operations in a centralized manner based on the disk's read/write settings to reduce disk fragments and track repeatedly to improve system performance. cache generally means caching, it is the cache memory for pages. It is a file system that stores data to be processed after being read from the disk. It saves the read data, when re-reading, if hit (find the required data), the hard disk will not be read. If it does not hit the hard disk, of course, the data is organized according to the read frequency, the most frequently read content will be placed in the easiest location to find;

The next line shows the memory usage of the application, and the two columns respectively show the actually used memory (the number allocated to the cache minus the number of unused buffers and cache, that is, the total amount of actually used buffers and cache) and the actual available memory of the system (the sum of unused buffers and cache and unallocated memory ), the "-/+" symbol indicates that the size of buffers/cache can be changed, rather than actually occupying the memory;

The last row of Swap displays the memory usage status of the Swap space, and the three columns respectively show the Swap capacity (total), usage (used) and available free Swap areas (free ); to use free for memory usage monitoring for a period of time, you can use the-s option to specify a time interval (unit: s) for continuous monitoring:

　　　　　　　　　　　　　　

2> use the vmstat command to fully monitor the memory:

To fully monitor the memory performance, run the vmstat command to display the status of physical memory and virtual memory, as well as CPU information:

The vmstat command monitors a large amount of data and is displayed in several categories:

The r and B columns of the procs (process) section show the number of processes that are ready to run and the number of processes that are in an uninterruptible sleep state, respectively, the so-called non-disruptive sleep state means that the process will not be awakened to a running state after receiving any signal, and will always wait for the hardware status to change;

The swpd, free, buff, and cache columns of memore (memory) show the virtual memory usage, idle physical memory, memory buffer, and cache respectively;

The si and so columns in the swap section show the number of bytes exchanged to and read from the disk per second;

The bi and bo columns in the I/O (input/output) section show the fast data read from Block devices and Block devices per second;

The in and cs columns in the system Section display the number of interruptions (including clock interruptions) and context switches (context switches) per second, respectively, when a process runs out of time slices or is preemptible by a process with a higher priority, it is forwarded to the waiting queue of the CPU and other processes are run on the CPU, the context switching process is called context switching. Too many switches will cause a great overhead of the system;

The cpu part shows the percentage of CPU usage time. The u table shows the user process time. sy indicates the system process time, id indicates the idle time, wa indicates the waiting time, And st indicates the time occupied by the virtual machine;

The vmstat command can also specify the data sampling interval and number of samples, that is, the following format is used:

Vmstat [sampling interval] [sampling times]

　　　　　　　　　　　　　　　　　　

 

4. Disk I/O performance monitoring:

Because the disk device is much slower than the CPU command processing speed, disk operations are the most slow operations in the whole process, although the disk's hardware technology, such as speed, the cache is constantly increasing, but the disk is still prone to system performance bottlenecks. The iostat tool can monitor system disk operation activities and report disk activity statistics, including data throughput and transmission requests. The basic usage is as follows:

Iostat [Option] [sampling interval] [number of samples]

　　　　　　　　　　　　

By default, the iostat command displays the summarized usage by disk Device (Device column) and CPU usage (avg-cpu part ), if option-d is added, only disk usage is counted. For disk data usage, the specific statistical data includes five items: tps indicates the number of I/O requests sent to the device per second; BLK_read/s and BLK_wrtn/s indicate the fast data read and written by the device per second, and BLK_read and BLK_wrtn indicate the total fast data read and written by the device; to view the usage of partitions on a disk, you can use the-p option to specify partitions. The option-t indicates the time displayed in each statistical result, if you want to change the disk usage unit block, use option-k to replace the block with KB, and-m to replace the block with MB;

In addition, you can use the sar-B command to calculate the I/O and transmission rate. By default, 10 minutes are used as an interval to display data from the last period of time. The statistics include five items: tps indicates the number of I/O requests from the physical disk per second (multiple logical requests will be merged into one I/O disk request, and the size of one transmission is uncertain ), rtps and wtps indicate the number of read requests and write requests per second, bread/s and bwrtn/s indicate the number of data records read from and written to the disk per second;

5. Comprehensive Monitoring through top:

The top command is an excellent Interactive performance monitoring tool. You can refresh the display including memory, CPU, process, and user data at specified intervals on a unified interface, the command format is as follows:

Top-hv |-bcHisS-d refresh interval-n refresh times-p pid [, pid...]

Option-p pid indicates that only the specified pid process information is displayed. The running result of the top command is as follows:

　　　　　　　　　　　　　　

Line 1 (top): displays the system running time, number of users, and average load;

Line 2 (Tasks): displays the summary of processes, including the total number of processes, number of running processes, and number of processes that are sleeping, number of stopped and dead processes;

Line 3 (CPU): displays the CPU usage percentage, namely, user processes, system processes, user processes that have changed their priorities, idle status, waiting for I/O, and hardware interruptions, CPU usage of software interruptions and virtual devices;

Row 4 (Mem): displays the physical memory information, which is the total physical memory, the number of used physical memory, the amount of unused physical memory, and the amount of memory used as the buffer;

Row 5 (Swap): displays the virtual memory information, which is the total virtual memory, the number of virtual memory used, the number of idle virtual memory, and the number of virtual memory used as the cache;

The last part is the performance statistics of each process. Each process has 12 items: PID (process ID), USER (USER who executes the process), PR (priority ), NI (nice value), VIRT (virtual memory size used by the process), RES (physical memory size used by the process), SHR (shared memory), S (Process status ), % CPU (CPU usage percentage), % MEM (percentage of physical memory used), TIME + (CPU usage TIME), COMMAND (process name );

6. Optimize system performance:

The basic steps for optimizing system performance are as follows:

1) use monitoring tools to monitor system activities;

2) analyze the obtained performance data to find out the links that cannot meet the performance requirements;

3) analyze the causes of performance degradation and take corresponding optimization measures;

A common way to improve system performance is to improve performance from hardware configuration. For example, you can use a SCSI interface hard disk instead of an IDE hard disk to create RAID using multiple hard disks and use the largest possible physical memory, use multiple processing systems;

If you only install Linux by default and do not configure the performance-related options in detail, the system performance will not be optimized, by adjusting the kernel, the overall system performance can be optimized. We recommend that you upgrade the kernel to a newer kernel version when adjusting the kernel. Generally, the new kernel version supports more performance options. to optimize the performance, you should take the initiative to discard features that consume too much resources based on your own needs, configure a custom kernel suitable for you;

The current high CPU usage of the system is caused by IO wait, not due to insufficient CPU resources. The user should check, adjust, and optimize the processes in which I/O operations are being performed in the system;

The low idle memory of the system does not necessarily indicate a systemic problem. This should be analyzed by combining the si and so indicators (page swap between memory and disk, when the physical memory can store data of all processes, physical memory and disk (Virtual Memory) should not have frequent page switching operations, only when the physical memory cannot meet the requirements will the system swap the data in the memory to the disk. Because the disk performance is much slower than the memory, if there is a large number of page exchanges, then the system performance will inevitably be greatly affected;