Linux testing cpu performance details

This article mainly introduces some basic knowledge about Linux cpu performance testing. first, we will introduce sched in Linux kernel. The scheduler is responsible for two types of resources in the scheduling system: thread and interrupt. The scheduler gives different priorities to different resources.

Linux testing cpu performance from high to low:
1. Hardware interruption Hardware Interrupts) -- these requests are triggered by Hardware, such as the disk has completed the read/write task or the NIC has received new packets.
2. Software interruption Software Interrupts) -- this refers to the kernel-state Software interruption that maintains kernel running. For example, the kernel clock management process.
3. real-time threads) -- a Real-time process has a higher priority than the kernel itself. It can seize the CPU time slice of the kernel. In 2.4, the kernel is an unpreemptible kernel, it does not support real-time programs.
4. Kernel process Kernel threads) -- including the Kernel program.
5. User threads) -- all processes running in the User State.

There are three important concepts about Linux cpu Performance Testing: context switch context switchs), Run queue) and utilization ).

Context switching:
Currently, popular CPUs can run only one thread at a time. hyperthreading processors can run multiple threads at a time, including multi-core CPUs ), the Linux kernel identifies a multi-core processor as multiple separate CPUs.
Why does a standard Linux Kernel support running 50 ~ 50000 processes run. For general CPUs, the kernel schedules and executes these processes. Each process runs on a CPU time slice. When a process runs out of time slice or is preemptible by a process with a higher priority, it is backed up to the CPU running queue, at the same time, other processes run on the CPU. This process is called context switching. Too many context switches may cause great overhead of the system.

Running queue:
Each CPU maintains a running queue. Ideally, the scheduler keeps running processes in the queue. The process is either in sleep or run able state. If the CPU is overloaded, the Scheduler cannot meet the requirements of the system. As a result, running processes will fill the queue. The larger the queue, the longer the program execution time. "Load" is used to indicate the running queue. With the top Command, we can see the size of the running queue in CPU for one minute, 5 minutes, and 15 minutes. The greater the value, the greater the system load.

Linux testing cpu performance CPU usage:
CPU usage can be divided into the following parts:
User Time-percentage of Time at which User processes are executed;
System Time-percentage of Time at which kernel processes are executed and interrupted;
Wait IO-percentage of time when the CPU is in the idle state due to IO Wait;
Idle-percentage of time when the CPU is in the Idle state.

About time slice and Dynamic Priority:
Time slice is a key parameter for the CPU. If the time slice is too long, the system's interaction will deteriorate and the user will not feel parallel. If it is too short, the system will frequently perform context switching, reducing the performance. IO Bound systems do not require too long time slice, because the system mainly involves IO operations. For CPU Bound systems, it requires a long time slice to maintain the effectiveness of the cache. The system will give a default priority when each process starts, but during the running process, the system will continuously adjust the priority according to the running status of the process, the kernel will increase or decrease the priority of the process each time increase or decrease by 5). The criteria are determined based on the time when the process is in sleep state. The IO Bound process is in sleep state most of the time, so the kernel will increase its priority, and the CPU Bound process will be punished by the kernel to reduce the priority. Therefore, if an I/O Bound process is run on a system and the CPU Bound process is run, we will find that the performance of the I/O Bound process will not decline, but the performance of the CPU Bound process will continue to decline.
We run a CPU Bound Program: cpu-hog. The ps command shows that its priority is declining.
 

 
 

  
  
term1# ./cpu-hog  
  
  
term2# while :; do ps -eo pid,ni,pri,pcpu,comm | egrep  
  
  
'hog|PRI'; sleep 1; done  
  
  
PID NI PRI %CPU COMMAND  
  
  
22855 0 20 84.5 cpu-hog  
  
  
PID NI PRI %CPU COMMAND  
  
  
22855 0 18 89.6 cpu-hog  
  
  
PID NI PRI %CPU COMMAND  
  
  
22855 0 15 92.2 cpu-hog  
  
  
PID NI PRI %CPU COMMAND  
  
  
22855 0 15 93.8 cpu-hog  
 
 

We run the find command, which is an IO Bound program. We can see that its priority is constantly increasing.
 

 
 

  
  
term1# find /  
  
  
term2# while :; do ps -eo pid,ni,pri,pcpu,comm | egrep  
  
  
'find|PRI'; sleep 1; done  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 20 0.0 find  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 21 4.0 find  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 23 3.5 find  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 23 4.3 find  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 23 4.2 find  
  
  
PID NI PRI %CPU COMMAND  
  
  
23101 0 23 4.4 find  
 
 

As we discussed earlier, the performance of any system is baseline-based, and the performance of CPU monitoring is the above three points. The running queue, CPU usage, and context switching are supported. The following are some common performance requirements for CPUs:
1. For each CPU, the running queue should not exceed 3. For example, if it is a dual-core CPU, it should not exceed 6;
2. If the CPU is running at full capacity, it should meet the following distribution,
A) User Time: 65% ~ 70%
B) System Time: 30% ~ 35%
C) Idle: 0% ~ 5%
3. Context switching depends on CPU usage. If the CPU usage meets the preceding distribution, a large number of context switches are acceptable.
Common monitoring tools include vmstat, top, dstat, and mpstat.
 

 
 

  
  
# vmstat 1  
  
  
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----  
  
  
r b swpd free buff cache si so bi bo in cs us sy id wa  
  
  
0 0 104300 16800 95328 72200 0 0 5 26 7 14 4 1 95 0  
  
  
0 0 104300 16800 95328 72200 0 0 0 24 1021 64 1 1 98 0  
  
  
0 0 104300 16800 95328 72200 0 0 0 0 1009 59 1 1 98 0  
 
 

R indicates the size of the running queue,
B indicates the number of threads in the block due to IO wait,
In indicates the number of interrupts,
Cs indicates the number of context switches,
Us indicates the user's CPU time,
Sys indicates the system CPU time,
Wa indicates the time when the CPU is in the idle state due to IO wait,
Id indicates the total time when the CPU is in the idle state.

To sum up, the Linux CPU performance monitoring test includes the following aspects:
Check the running queue of the system to ensure that the running queue of each CPU is no greater than 3.
Ensure that the CPU usage distribution meets the 70/30 principle: 70% of users, 30% of the system ).
If the system takes too long, it may be because of frequent scheduling and priority changes.
CPU Bound processes are always penalized to lower the priority) while IO Bound processes are always rewarded to increase the priority ).