Linux multicore bound hardware interrupts to different CPUs

Hardware interrupts occur frequently, is a very CPU-intensive thing, in multi-core CPU conditions if there is a way to allocate a large number of hardware interrupts to different CPU (core) processing is clearly a good balance of performance. Now the server is still multi-CPU multi-core, multi-card, multi-disk, if you can interrupt the network card exclusive 1 CPU (CORE), Disk IO interrupt exclusive 1 CPUs will greatly reduce the burden of a single CPU, improve the overall processing efficiency. Vpsee the day before yesterday received a Netizen's email mentions the SMP IRQ Affinity, caused the topic today: D, the following operations in the SUN fire X2100 M2 Server + 64-bit version of CentOS 5.5 + Linux 2.6.18-194.3.1.el5 On the implementation.

What is an interrupt

The definition of "interruption" in Chinese textbooks is too blunt, simply to say:

Each hardware device (such as: Hard disk, network card, etc.) need to have some form of communication with the CPU so that the CPU can know what happened in time, so the CPU may put down the matter to deal with the emergency, the hardware device actively disturb the CPU phenomenon can be called hardware interruption.

Just as you are working at the time of QQ interference, a QQ shake can be called an interruption.

Interrupts are a good way to communicate between CPUs and hardware.

There is also a way to call polling (polling):

is to let the CPU timing the hardware status of the query and then do the corresponding processing.

It's like you have to check QQ every 5 minutes to see if anyone is looking for you, is this a waste of your (CPU) time? So interrupts are a hardware-active approach that is more efficient than polling (CPU active).

OK, here's another problem, each hardware device is interrupted, so how to distinguish between different hardware?

How do different devices interrupt at the same time how to know which interrupt is from the hard disk and which is from the NIC? This is very easy, not every QQ number is not the same?

Similarly, each hardware device on the system is assigned an IRQ number, which distinguishes Zhang San from Li Four by this unique IRQ number.

In a computer, an interrupt is an electrical signal that is generated by the hardware and sent directly to the interrupt controller (such as 8259A) and then signaled to the CPU by the interrupt controller, and the CPU detects the signal and interrupts the current work in turn to handle the interrupt.

The processor then notifies the operating system that an interrupt has been generated so that the operating system handles the interrupt appropriately.

Now take a look at the interrupt controller, there are two common interrupt controllers: Programmable Interrupt controller 8259A and Advanced Programmable Interrupt Controller (APIC), interrupt controller should be taught in the university's hardware interface and computer architecture related courses.

Traditional 8259A is only suitable for single CPU, now is multi-CPU multi-core SMP system, so in order to take full advantage of the SMP architecture, the interruption to each CPU on the system to better achieve parallelism and improve performance, Intel introduced the Advanced Programmable Interrupt Controller (APIC).

The hardware support of the Advanced Programmable interrupt controller is not enough, the Linux kernel must also be able to take advantage of these hardware characteristics, so only kernel 2.4 later support the different hardware interrupt request (IRQS) allocated to the specific CPU, this binding technology is called SMP IRQ Affinity.

For more information, see the documentation for the Linux kernel source code: Linux-2.6.31.8/documentation/irq-affinity.txt

How to use

Let's see how the interrupts on the system are distributed on the CPU, and it's clear that the CPU0 is dealing with more interrupts:

Cat/proc/interrupts

           CPU0       CPU1         0:  918926335          0    IO-APIC-edge  timer  1:          2          0    IO-APIC-edge  i8042  8:          0          0    IO-APIC-edge  rtc  9:          0          0   IO-APIC-level  acpi 12:          4          0    IO-APIC-edge  i8042 14:    8248017          0    IO-APIC-edge  ide0 50:        194          0   IO-APIC-level  ohci_hcd:usb2 58:      31673          0   IO-APIC-level  sata_nv 90:    1070374          0         PCI-MSI  eth0233:         10          0   IO-APIC-level  ehci_hcd:usb1NMI:       5077       2032 LOC:  918809969  918809894 ERR:          0MIS:          0

In order not to let CPU0 very tired how to transfer partial interruption to CPU1?

Or how to transfer the interrupt of eth0 NIC to CPU1?

Check the SMP affinity for IRQ 90 interrupts first to see how the current interrupts are allocated on different CPUs (FFFFFFFF means allocated on all available CPUs):

# cat /proc/irq/90/smp_affinity 7fffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff

Before we go any further, we need to stop the IRQ auto-tuning process so that we can manually bind the IRQ to different CPUs, otherwise the custom-made changes will be overwritten by the auto-tuning process.

If you want to modify the interrupt handling for IRQ 90, bind to the 2nd CPU (CPU1):

# /etc/init.d/irqbalance stop# echo "2" > /proc/irq/90/smp_affinity

(How did the echo "2" come from above?) Why is "2"?

Please refer to this: Calculate SMP IRQ Affinity) over a period of time in the see/proc/interrupts, is not 90:eth0 on the CPU1 interrupt increased (145), the interruption on the CPU0 has not changed?

Printing/proc/interrupts will find that the number of interrupts eth0 on the CPU0 remains constant, and the number of interrupts on the CPU1 is constantly increasing, which is exactly what we want:

# cat /proc/interrupts            CPU0       CPU1         0:  922506515          0    IO-APIC-edge  timer  1:          2          0    IO-APIC-edge  i8042  8:          0          0    IO-APIC-edge  rtc  9:          0          0   IO-APIC-level  acpi 12:          4          0    IO-APIC-edge  i8042 14:    8280147          0    IO-APIC-edge  ide0 50:        194          0   IO-APIC-level  ohci_hcd:usb2 58:      31907          0   IO-APIC-level  sata_nv 90:    1073399        145         PCI-MSI  eth0233:         10          0   IO-APIC-level  ehci_hcd:usb1NMI:       5093       2043 LOC:  922389696  922389621 ERR:          0MIS:          0

What's the use?

In the case of the network is very heavy, for the file server, high-traffic Web server such applications, the different network card IRQ equalization to different CPUs will reduce the burden on a certain CPU, improve the ability of multiple CPU overall processing interrupt;

For an application such as a database server, tying the disk controller to a CPU and binding the NIC to another CPU will improve the response time of the database and optimize performance. A reasonable balance of IRQ interrupts based on your production environment and application characteristics can help improve the overall throughput and performance of the system.

Vpsee often received a letter asked how to optimize the Linux, the optimization of the VPS, the problem is not very good answer, to remember is that performance optimization is a process rather than the result, not to see some of the document changed the parameter is called optimization, the back also need a lot of testing, monitoring and continuous observation and improvement.

?

Linux multicore bound hardware interrupts to different CPUs