Nic binding kernel

Nic binding to a single kernel process and asynchronous I/O should be able to achieve optimal communication performance, but in reality, we often find that this mode does not achieve the expected results, this may be because the NIC is competing for CPU resources with the application. The frequent occurrence of hardware interruptions is a very CPU resource-consuming task... the single process and asynchronous I/O of the kernel bound to the NIC should be able to achieve optimal communication performance. However, in reality, we often find that this mode does not achieve the expected results, this may be because the NIC is competing for CPU resources with the application. Frequent hardware interruptions consume CPU resources, if there is a way to allocate a large number of hardware interruptions to a specific CPU core for processing, you can achieve better performance. The current servers are mostly multi-CPU, multi-core, multi-Nic, and multi-hard disk. if each interruption can be dispersed and balanced, the binding of specific hardware is interrupted to a specific CPU core, for example, if the Nic interrupt excludes one CPU core and the disk I/O interrupt excludes one CPU core, it will greatly reduce the burden on a single CPU and improve the overall processing efficiency. 1. what is interruption? The definition of "interrupt" in Chinese textbooks is too rigid. Simply put, each hardware device (such as a hard disk or network card) needs to communicate with the CPU in some form, so that the CPU can know what has happened in time, so that the CPU may put down things in its hands to handle emergency incidents. the active interruption of hardware devices to the CPU can be called a hardware interruption. Just like QQ interference when you are working, a QQ avatar flash can be understood as being interrupted. Interruption is a good way to communicate between the CPU and hardware. Another method is Polling, which allows the CPU to periodically query the hardware status and perform corresponding processing, it's like checking QQ every five minutes to see if anyone is looking for you. Is this a waste of time? Therefore, interruption is an active hardware method, which is more effective than polling (CPU active. There is another problem here. every hardware device is interrupted. how can we differentiate different hardware? How do I know which one is from the hard disk and which one is from the NIC when the device is interrupted at the same time? This is actually very easy, as if every QQ number is different. Similarly, the system will assign an IRQ number to each hardware device, this unique IRQ number can be used to differentiate different hardware. In a computer, interruption is an electrical signal generated by hardware and directly sent to the interrupt controller. then, the interrupt controller sends a signal to the CPU. after the CPU detects the signal, in this case, the current job is interrupted and the interrupted job is processed. Then, the processor will notify the operating system that an interruption has occurred, so that the operating system will handle the interruption as appropriate. Now let's take a look at the interrupt controller. There are two common types of interrupt controllers: Programmable Interrupt Controller 8259A and Advanced Programmable Interrupt Controller (APIC ). The traditional 8259A is only suitable for a single CPU. it is now a multi-CPU, multi-core SMP system, so in order to make full use of the SMP architecture, intel introduced an advanced programmable interrupt controller (APIC) to deliver interruptions to each CPU in the system for better parallel performance and performance improvement ). Hardware support for advanced programmable interrupt controllers is not enough, and the Linux kernel must be able to take advantage of these hardware features. Therefore, only versions later than kernel 2.4 support different hardware interrupt requests (IRQs) allocated to a specific CPU core. this binding technology is called smp irq Affinity. For more information, see the Linux kernel source code Documentation: linux-2.6.31.8/Documentation/IRQ-affinity.txt. 2. how to use it? First, let's take a look at two basic commands: cat/proc/interrupts and check the system interruption. generally, Nic interruption is allocated to CPU0. Cat/proc/cpuinfo: check the CPU information, CPU, and core. Then we run the command to check how the system interrupts are allocated to the CPU. Obviously, there are more interruptions to be processed on CPU0: # cat/proc/interrupts CPU0 CPU1 0: 1: 8: 9: 12: 14: 50: 58: 90: 233: NMI: LOC: ERR: MIS: 918926335 2 0 0 4 8248017 194 31673 10 1070374 5077 0 0 0 0 0 0 0 0 0 0 0 0 0 918809969 2032 0 0 IO-APIC-edge IO- APIC-edge IO-APIC-level IO-APIC-edge IO-APIC-level PCI-MSI IO-APIC-level timer i8042 rtc acpi i8042 id E0 ohci_hcd: usb2 sata_nv eth0 ehci_hcd: to prevent excessive loads on CPU0, how can I transfer some interruptions to CPU1? Or how to switch the interruption of the eth0 Nic to CPU1? First, we need to check the smp affinity interrupted by IRQ 90 (that is, the IRQ number of eth0 NIC, check how the current interrupt is allocated to different CPUs (ffffffff means it is allocated to all available CPUs): # cat/proc/irq/90/smp_affinity 7 fffffffff, ffffffff, ffffffff, ffffffff, and ffffffff before further operations, we need to stop the service process automatically adjusted by IRQ so that we can manually bind IRQ to different CPUs, otherwise, the changes made by manually binding will be overwritten by the automatic adjustment process. To modify the interrupt processing of IRQ 90 and bind it to 2nd CPUs (CPU1), run the following command: #/etc/init. d/irqbalance stop # echo "2">/proc/irq/90/smp_affinity the "2" in "echo 2>/proc/irq/90/smp_affinity" is how. This is actually a binary number, representing 00000010. if 00000001 represents CPU0, then 00000010 represents CPU1, "echo 2>/proc/irq/90/smp_affinity" means to bind a 90 interrupt to 00000010 (CPU1. Therefore, each CPU can be expressed in binary or hexadecimal format: binary Hex CPU 0 00000001 1 CPU 1 00000010 2 CPU 2 00000100 4 CPU 3 00001000 8 if you want to bind IRQ to CPU2 (that is, 00000100 = 4), run the following command: # echo "4">/proc/irq/90/smp_affinity if you want to balance IRQ to CPU0 and CPU2 at the same time, that is, 00000001 + 00000100 = 00000101 = 5, then execute the following command: # echo "5">/proc/irq/90/smp_affinity there is also a limit that IO-APIC has two working modes: logic and physical. In logic mode, the IO-APIC can distribute the same interrupt to eight CPU cores at the same time (limited by bitmask registers because bitmask is only 8 bits); in physical mode, the same type of interruption cannot be distributed to different CPU cores at the same time. for example, the eth0 interruption cannot be handled by CPU 0 and CPU 1 at the same time. in this case, only eth0 to CPU 0 can be located, and eth1 to CPU 1 can be located, that is, the eth0 interrupt cannot be processed by multiple CPU cores at the same time as the logic mode. After a period of time, check the/proc/interrupts information and find that the 90: eth0 interrupt on CPU1 increased by 145 times, continuously print/proc/interrupts information, we will find that the number of eth0 interruptions on CPU0 remains unchanged, and the number of interruptions on CPU1 continues to increase. this is what we want: # cat/proc/interrupts CPU0 CPU1 0: 1: 8: 9: 12: 14: 50: 58: 90: 233: NMI: LOC: ERR: MIS: 922506515 2 0 0 4 8280147 194 31907 10 1073399 0 0 0 0 0 0 0 0 0 0 0 5093 0 922389696 145 0 0 IO-APIC-edge IO- APIC-edge IO-APIC-level IO-APIC-edge IO- APIC-edge IO-APIC-level PCI-MSI IO-APIC-level timer i8042 rtc acpi i8042 ide0 ohci_hcd: usb2 sata_nv eth0 ehci_hcd: usb1 3, what is the use? When the network load is very heavy, for applications such as file servers and high-traffic Web servers, the IRQ of different NICs is evenly bound to different CPU cores, this reduces the burden on a single CPU and improves the overall interrupt handling capability of multiple CPUs and cores. For applications such as database servers, binding a disk controller to a CPU core and a Nic to another CPU core will increase the database response time and optimize the performance. Balancing IRQ interruptions based on your production environment and application characteristics helps improve the overall throughput and performance of the system.