The interrupt subsystem of Linux kernel (a): a review

First, preface

A qualified Linux driver engineer needs to have a deep understanding of the interrupt subsystem in the kernel, only in this way, when writing a specific driver:

1. Correct use of the API provided by Linux kernel, such as the most famous REQUEST_THREADED_IRQ (REQUEST_IRQ) interface

2, correct use of synchronization mechanism to protect the critical section of the driver code

3, the correct use of kernel provided by the SOFTIRQ, Tasklet, Workqueue and other mechanisms to complete the specific interrupt processing

For the reasons above, I would like to be able to describe in a series of documents the knowledge of all aspects of the interrupt subsystem in Linux kernel. On the one hand is to organize your own thoughts, on the other hand, I hope to be able to other driver engineers (or want to engage in Linux-driven work of engineers) have some help.

Second, interrupt system related hardware description

Interrupt hardware system mainly has three kinds of device participation, each peripheral, interrupt controller and CPU. Each peripheral provides an IRQ request line, which, when an interrupt event occurs, requests the processing of the CPU system via an electrical signal on the IRQ request line. The peripheral IRQ request line is too many, the CPU needs a small partner to help him, this is the interrupt controller. The Interrupt controller is a bridge that connects peripheral interrupt systems and CPU systems. Depending on how much the peripheral IRQ request line is, the Interrupt controller can cascade. The main function of the CPU is the operation, so the CPU does not handle the interrupt priority, which is the interrupt controller thing. For CPUs, there are generally two kinds of interrupt requests, such as: for ARM, IRQ and FIQ signal lines, respectively, let arm into IRQ mode and Fiq mode. For X86, there is a shielded interrupt and a non-shielded interrupt.

This section does not describe the specific hardware, but rather uses the concept of HW block. For example, the CPU HW block is a logical description of the actual hardware block such as arm core or X86, which may be any CPU block in practice.

1. Logical block diagram of HW interrupt system

My understanding of the logical block diagram of the HW interrupt system is as follows:

There are several CPU blocks in the system to receive interrupt events and processing, several interrupt controllers form a tree-like structure, assemble the IRQ request line for all peripherals in the system, and distribute the interrupt event to a CPU block for processing. From the interface level, there are two main types of interfaces, one is interrupt interface. In some implementations, the specific interrupt interface is the form of a hardware signal line, through the level signal transmission interrupt event (ARM and GIC composed of the interrupt system is so designed). Some systems use other methods to pass interrupt events, such as the X86+apic (Advanced Programmable Interrupt Controller) system, each x86 core has a local APIC, these local The APIC is connected to the IO APIC via the ICC (Interrupt Controller communication) bus. The IO APIC collects interrupts from each peripheral and translates it into a message on the bus that is passed to the local APIC on a CPU. As a result, the red line above is also a logic-level interrupt signal, possibly a copper wire on the actual PCB (or a copper wire inside the SOC), or it may be a message. In addition to interrupting the interface, there is a need for control information exchange between the CPU and the interrupt controller. The Interrupt controller will open some registers for the CPU to access and control.

2. Topology between multiple interrupt controllers and multiple CPUs

The Interrupt controller has some support for multiple CPU cores (such as GIC, APIC, etc.), some not supported (such as s3c2410 interrupt Controller, X86 platform pic, etc.). If there is only one GIC in the hardware platform, all peripheral interrupts can be distributed to the CPU connected to the interrupt controller by means of a register that controls the GIC. What if there are multiple GIC (or the cascaded Interrupt controller supports multi CPU core)? Suppose we want to design a very complex system, there are 8 CPUs in the system, 2000 peripherals interrupt to be processed, how do you design the interrupt controller in the system? If you use GIC, we need two GIC (one GIC supports up to 1024 interrupt sources), one is root GIC and the other is the secondary GIC. In this case, you have two options:

(1) The 8 CPUs are connected to the root GIC, secondary GIC does not receive the CPU. The peripheral interrupt that was originally hooked up to secondary GIC is output to a CPU, and now it can only be (via a CPU interface IRQ signal) to one of the SPI on root GIC. For software, this is a relatively simple design, secondary GIC's CPU interface settings are fixed, always from a fixed CPU interface output to the root GIC. The downside to this scenario is that the PPI and SGI at this time secondary GIC are useless. In addition, in this setting, all peripheral interrupts connected to the secondary GIC are processed uniformly by the target CPU, either sent to Cpu0 or CPU 5, and cannot be controlled separately.

(2) Of course, you can also have each GIC connected to 4 CPU Core,root GIC connection cpu0~cpu3,secondary GIC connection Cpu4~cpu7. In this state, the interrupts connected to the root GIC can be handled by CPU0~CPU3, and interrupts connected to the secondary GIC can be handled by the CPU4~CPU7. In this way, there is no 8-core power to be seen in the interruption process.

Note: The logical block in the previous section uses scheme one.

3. What CPU does the Interrupt controller send the interrupt event to?

There is no doubt that only the interrupt controller that supports multi CPU core has this happiness problem. In general, an interrupt controller can escalate an interrupt event to a CPU or a group of CPUs (including broadcasting to all CPUs). The interruption of the peripheral type, of course, is sent to a CPU on the OK, I do not see the need to send such interrupts to multiple CPUs for processing. If multiple CPUs are delivered, in fact, there should be only one handler actually interacting with the peripherals, and the other handler on the CPU should look like this: the interrupt that the IRQ number corresponds to has been processed by another CPU, and exits the handler directly, Return to the interrupted scene. The IPI interrupt does not have this limitation, and the IPI is more like a mechanism for communication between CPUs, which should be without pressure on such interrupted broadcasts.

In fact, from a user's point of view, the requirements are quite complex, and our goals may include:

(1) An interrupt for an IRQ number is handled by a particular CPU

(2) allow a specific interrupt to be handled by several CPUs in turn

......

Of course, the specific requirements may be more complex, but how to distinguish between software and hardware division of labor? It is unreasonable to let the hardware handle the complex strategy, if the complex logic is implemented by hardware, it means more transistors and more power. Therefore, the most common practice is to set a target CPU control interface for each interrupt supported by the interrupt controller (it should certainly be in register form, for GIC, this register is interrupt processor target Register). The system has multiple CPUs, the control interface has a number of bits, each bit represents a CPU. If the bit is set to 1, then the interrupt is escalated to the CPU, and if 0, it is not reported to the CPU. This kind of hardware logic is relatively simple, the rest of the control content is given to the software. For example, if the system has two CPU cores, an interrupt wants to be processed by two CPUs in turn. Then when the CPU0 phase should break into the interrupt handler, the interrupt processor target register in the corresponding bit set to 0, the other CPU bit set to 1. The Interupt controller then sends the interrupt to CPU1 when the next outage occurs. For CPU1, the CPU0 bit in interrupt processor target register is set to 1,disable the bit of this CPU when the handler of the interrupt is executed, so that when the next interrupt occurs, Interupt Controller gave the interruption to CPU0. The result of this software control is to realize the algorithm that the specific interrupt is handled by 2 CPUs in turn.

4. More Thinking

In the face of the logic block diagram of this HW interrupt system, we can actually ask more questions:

(1) is the interrupt controller sent to the CPU to be recoverable? Redistribute to another CPU?

(2) How can interrupts in the system be distributed for better performance?

(3) What are the factors that need to be considered in the strategy of interrupting distribution?

......

Many problems actually I have no answer, slowly thinking, slowly approaching the truth.

II. software framework related to interrupt subsystem

The software framework diagram for the Linux kernel interrupt subsystem is as follows:

From the block diagram above, we know that the Linux kernel interrupt subsystem is divided into 4 parts:

(1) Hardware-independent code, which we call the Linux kernel General interrupt processing module. No matter what kind of CPU, which controller, where the process of processing has some of the same content, the same content is abstracted out, and HW Independent. In addition, the driver code for each peripheral also wants to be able to implement IRQ-related management with a single interface (not related to specific interrupt hardware systems and CPU architectures) These "generic" codes form the core part of the Linux kernel interrupt subsystem.

(2) CPU architecture related interrupt processing. is related to the specific CPU architecture used by the system.

(3) Interrupt controller driver code. is related to the interrupt controller used by the system.

(4) Common peripheral driver. These drivers will implement their own driver logic using the API of the Linux kernel General interrupt processing module.

Third, interrupt subsystem document planning

The interruption-related documentation is planned as follows:

1, Linux kernel interrupt subsystem (a), that is, this article, in fact, is an introduction, no actual content, mainly to the reader a general framework of hardware and software.

2, Linux kernel interrupt subsystem (ii): IRQ Domain Introduction. The main description is how to convert a HW interrupt ID into an IRQ number.

3. Linux Kernel interrupt Subsystem (III): IRQ number and interrupt descriptor. It mainly describes the data structure and interface APIs associated with interrupt descriptors.

4, Linux kernel interrupt subsystem (FOUR): High level IRQ event handler.

5, Linux kernel interrupt subsystem (v): Driver API. Mainly with a common driver for the perspective of the Linux interrupt subsystem provides the API, how to use these APIs, allocate resources, whether resources, how to deal with interrupt-related synchronization problems and so on.

6, Linux kernel interrupt subsystem (VI): ARM Interrupt processing process, this document takes arm CPU as an example, describes arm related interrupt processing process

7, Linux kernel interrupt subsystem (VII): GIC Code Analysis, this document is a specific interrupt Controller For example, describes the code composition of the IRQ chip driver.

8. Linux kernel interrupt subsystem (eight): SOFTIRQ

9, Linux kernel interrupt subsystem (ix): Tasklet

The interrupt subsystem of Linux kernel (a): a review