About Linux system command top%si High occupancy, analysis example one

"Top%wa Problem", then encountered top %si too high (more than 95% peak time) problem.

%wa high, indicating disk is busy. For example, disk read and write times are very high.
%si High, does it indicate soft interrupt busy? Does it also indicate that the number of soft interrupts is very high?

In order to verify this problem, some tools for Linux monitoring interrupt have been found. At first find Oprofile, it is very powerful, strong to use up to touch the north. Can monitor hard interrupts with oprofile, but when the%si is high, the hard interrupt is monitored to zero.
Then find the ProcInfo can monitor the detailed soft interrupt, can appear the device and the corresponding number of interrupts. The two%si high project monitoring results are as follows:
----------------------------------------------------------------------
Project A, under high pressure scenario execution 2 minutes:

before testing: (Query front-end [Server C] data when no tests are performed)
context:11608691780
IRQ 0:377348704 Timer
IRQ 14:41845647 IDE0
IRQ 58:2,557,376,548 0
irq233:69455022 ioc0

Problem Environment: (interface configured at [Server G], query front-end [Server C] data)
context:11609397277 difference: 705497
IRQ 0:377651500 Timer difference: 302796
IRQ 14:41848329 ide0 Difference: 2682
IRQ 58:2,557,728,212 0 Difference value: 351664
irq233:69457769 ioc0 Difference Value: 2747

Normal environment: (interface configured at [Server E], query front-end [Server C] data)
context:11611152447 difference: 1755170
IRQ 0:378089297 Timer difference: 437797
IRQ 14:41852253 ide0 Difference: 3924
IRQ 58:2,558,490,578 0 Difference value: 762366
irq233:69506181 ioc0 Difference Value: 48412

----------------------------------------------------------------------

Project B, under high pressure scenario execution 2 minutes:

Before testing: (data from server E) When testing is not performed
context:365586128
IRQ 0:555887117 Timer
IRQ 14:4970216 IDE0
IRQ 98:6.6732 million 0
irq233:2963048 ioc0

Normal Environment: (Database connection pool set to localhost)
context:368023507 difference: 2437379
IRQ 0:556101285 Timer difference: 214168
IRQ 14:4972124 ide0 Difference: 1908
IRQ 98:7,013,268 0 Difference value: 340068
irq233:2968985 ioc0 Difference Value: 5937

Problem Environment: (The database connection pool is set to the native IP address)
context:369679349 difference: 1655842
IRQ 0:556446790 Timer difference: 345505
IRQ 14:4975148 ide0 Difference: 3024
IRQ 98:7,298,545 0 Difference value: 285277
irq233:2974283 ioc0 Difference Value: 5298

----------------------------------------------------------------------

Normal environment, refers to the%si value fluctuation within the normal range. Problem environment refers to%si Anomaly, General%si will reach 70%~95%. The difference is the number of interrupts that the corresponding device produces during the test.
The number of interrupts in a normal environment is higher than the number of interrupts in an abnormal situation. Description
Computers have the ability to handle more interrupts in a unit, and the number of interrupts does not represent%si high.
It is further explained that the%si high means that the CPU waits for a soft interrupt to complete at a higher time.
Combined with the actual situation of Project B, the MySQL connection mode of lamp is set to permanent connection (long connection),%si will not be high:
1. instructions php in connection with MySQL in the use of localhost connection does not go to the network, process switching between the effort, the CPU does not have to wait;
2. While PHP is connected using an IP address, switching between processes consumes resources, causing the CPU to wait;
3. Inter-process switching creates interrupts, and the CPU waits for the CPU to wait for the soft interrupt processing to complete.

There has been no theory to prove the above, temporarily through the phenomenon of inference, hope that a day can be confirmed by the theory.

About Linux system command top%si High occupancy, analysis example one