Linux multicore bound hardware interrupts to different CPUs (IRQ Affinity) turn

Hardware interrupts occur frequently, is a very CPU-intensive thing, in multi-core CPU conditions if there is a way to allocate a large number of hardware interrupts to different CPU (core) processing is clearly a good balance of performance. Now the server is still multi-CPU multi-core, multi-card, multi-disk, if you can interrupt the network card exclusive 1 CPU (CORE), Disk IO interrupt exclusive 1 CPUs will greatly reduce the burden of a single CPU, improve the overall processing efficiency. Vpsee the day before yesterday received a Netizen's email mentions the SMP IRQ Affinity, caused the topic today: D, the following operations in the SUN fire X2100 M2 Server + 64-bit version of CentOS 5.5 + Linux 2.6.18-194.3.1.el5 On the implementation.

What is an interrupt

Chinese textbooks on the definition of "interrupted" too blunt, simply said, each hardware device (such as: Hard disk, network card, etc.) need to have some form of communication with the CPU so that the CPU in time to know what happened, so that the CPU may put down the matter to deal with emergency events, hardware devices actively interrupt the CPU The phenomenon can be called hardware interruption, as you are working at the time of QQ interference, the same time QQ shake can be called interruption.

Interrupt is a good way of CPU and hardware communication, there is also a way called polling (polling), is to let the CPU timing the hardware status and then do the corresponding processing, as if you every 5 minutes to check QQ to see if there are people looking for you, Is this way of wasting your (CPU) time? So interrupts are a hardware-active approach that is more efficient than polling (CPU active).

OK, here's another problem, each hardware device is interrupted, so how to distinguish between different hardware? How do different devices interrupt at the same time how to know which interrupt is from the hard disk and which is from the NIC? This is very easy, not every QQ number is not the same? Similarly, each hardware device on the system is assigned an IRQ number, which distinguishes Zhang San from Li Four by this unique IRQ number.

In a computer, an interrupt is an electrical signal that is generated by the hardware and sent directly to the interrupt controller (such as 8259A) and then signaled to the CPU by the interrupt controller, and the CPU detects the signal and interrupts the current work in turn to handle the interrupt. The processor then notifies the operating system that an interrupt has been generated so that the operating system handles the interrupt appropriately. Now take a look at the interrupt controller, there are two common interrupt controllers: Programmable Interrupt controller 8259A and Advanced Programmable Interrupt Controller (APIC), interrupt controller should be taught in the university's hardware interface and computer architecture related courses. Traditional 8259A is only suitable for single CPU, now is multi-CPU multi-core SMP system, so in order to take full advantage of the SMP architecture, the interruption to each CPU on the system to better achieve parallelism and improve performance, Intel introduced the Advanced Programmable Interrupt Controller (APIC).

The hardware support of the Advanced Programmable interrupt controller is not enough, the Linux kernel must also be able to take advantage of these hardware characteristics, so only kernel 2.4 later support the different hardware interrupt request (IRQS) allocated to the specific CPU, this binding technology is called SMP IRQ Affinity. For more information, see the documentation for the Linux kernel source code: Linux-2.6.31.8/documentation/irq-affinity.txt

How to use

Let's see how the interrupts on the system are distributed on the CPU, and it's clear that the CPU0 is dealing with more interrupts:

# cat/proc/interrupts            CPU0       CPU1         0:  918926335          0    io-apic-edge  timer  1:          2          0    Io-apic-edge  i8042  8:          0          0    io-apic-edge  RTC  9:          0          0   Io-apic-level  ACPI:          4          0    io-apic-edge  i8042:    8248017          0    Io-apic-edge  ide0:        194          0   io-apic-level  ohci_hcd:usb2:      31673          0   Io-apic-level  Sata_nv:    1070374          0         pci-msi  eth0233:          0   io-apic-level  ehci_ Hcd:usb1nmi:       5077       2032 LOC:  918809969  918809894 ERR:          0MIS:          0

In order not to let CPU0 very tired how to transfer partial interruption to CPU1? Or how to transfer the interrupt of eth0 NIC to CPU1? Check the SMP affinity for IRQ 90 interrupts first to see how the current interrupts are allocated on different CPUs (FFFFFFFF means allocated on all available CPUs):

# cat/proc/irq/90/smp_affinity 7FFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF,FFFFFFFF

Before we go any further, we need to stop the IRQ auto-tuning process so that we can manually bind the IRQ to different CPUs, otherwise the custom-made changes will be overwritten by the auto-tuning process. If you want to modify the interrupt handling for IRQ 90, bind to the 2nd CPU (CPU1):

#/etc/init.d/irqbalance stop# echo "2" >/proc/irq/90/smp_affinity

(How did the echo "2" come from above?) Why is "2"? Please refer to this: Calculate SMP IRQ Affinity) over a period of time in the see/proc/interrupts, is not 90:eth0 on the CPU1 interrupt increased (145), the interruption on the CPU0 has not changed? Printing/proc/interrupts will find that the number of interrupts eth0 on the CPU0 remains constant, and the number of interrupts on the CPU1 is constantly increasing, which is exactly what we want:

# cat/proc/interrupts            CPU0       CPU1         0:  922506515          0    io-apic-edge  timer  1:          2          0    Io-apic-edge  i8042  8:          0          0    io-apic-edge  RTC  9:          0          0   Io-apic-level  ACPI:          4          0    io-apic-edge  i8042:    8280147          0    Io-apic-edge  ide0:        194          0   io-apic-level  ohci_hcd:usb2:      31907          0   Io-apic-level  Sata_nv:    1073399        145         pci-msi  eth0233:          0   io-apic-level  ehci_ Hcd:usb1nmi:       5093       2043 LOC:  922389696  922389621 ERR:          0MIS:          0

What's the use?

In the case of the network is very heavy, for the file server, high-traffic Web server such applications, the different network card IRQ equalization to different CPUs will reduce the burden on a certain CPU, improve the overall processing of multiple CPUs the ability to interrupt; For applications such as database servers, the disk Binding the controller to one CPU and tying the NIC to another CPU will increase the response time of the database and optimize performance. A reasonable balance of IRQ interrupts based on your production environment and application characteristics can help improve the overall throughput and performance of the system.

Vpsee often received a letter asked how to optimize the Linux, the optimization of the VPS, the problem is not very good answer, to remember is that performance optimization is a process rather than the result, not to see some of the document changed the parameter is called optimization, the back also need a lot of testing, monitoring and continuous observation and improvement.

Linux multicore bound hardware interrupts to different CPUs (IRQ Affinity) turn