Linux Kernel Analysis notes-upper and lower half (upper)

Hi, if you have a good memory, I mentioned in my previous blog that interrupt processing is divided into two parts: the interrupt processing program is the upper half, it receives an interruption and immediately executes it, but only performs work with a strict time limit. In addition, it is called another part of the lower half to do the work that can be completed later. The lower half is the focus of today.

The lower half of the task is to execute a task that is closely related to the interrupt processing but does not execute the interrupt processing program itself. The best case is, of course, that the interrupt handler gives all the work to the lower half for execution, and does nothing on its own. Because we always want to interrupt the handler to return as quickly as possible. However, the interrupt handler is destined to do part of the work. Unfortunately, no one strictly specifies which part of the task should be completed. In other words, this decision is entirely made by a driver engineer like ours. Remember, the interrupt handler will be executed asynchronously and, in the best case, it will lock the current disconnection, so the smaller the interrupt handler, the better. Of course, having no rules does not mean having no experience or lessons:

1. If a task is very time-sensitive, it is a good choice to tell me whether to put it in the interrupt handler for execution. 2. If a task is related to hardware, place it in the interrupt handler for execution. 3. To ensure that a task is not interrupted by other interruptions (especially the same interruptions), put it in the interrupt processing program. 4. All other tasks, unless you have a better reason, are all thrown to the lower half for execution.

In a word, the sooner the interrupt handler is executed, the better.

The lower half of the old half will be executed later. What is the concept in the future? Unfortunately, this is just relative to the instant. In the lower half, you need to specify a specific time, as long as you delay the task a little, so that they can execute it after the system is not busy and the recovery is interrupted. Generally, the lower half will be executed immediately after the interrupt handler has returned. The key is to allow all corresponding interruptions when they are running.

The upper half can only be completed by the interrupt processing program, but there are many ways to implement the lower half. The mechanisms used to implement the lower half are composed of different interfaces and subsystems. The earliest was "bottom half". This mechanism is also called "BH". It provides a simple interface and provides a static creation list consisting of 32 bottom half, the upper half identifies the bottom half Executable by one of the 32-bit integers. Each BH is synchronized globally. Even if it is a different processor, no two bottom half instances can be executed simultaneously. This method is convenient but not flexible enough, but has a performance bottleneck. To require a better method. The second method is task queue. The kernel defines a group of queues. Each queue contains a linked list composed of functions waiting for calling. Depending on the location of the queue, these functions will be executed at a certain time point, and the driver can register their lower half to the appropriate queue as needed. This method is good but not flexible enough. It cannot replace the entire BH interface. Subsystems with high performance requirements, such as the network, cannot be competent either. In the 2.3 development version, softirqs and tasklet are introduced. Here, the soft interrupt and the Soft Interrupt mentioned in implementing system calls are not the same concept. If you do not need to consider compatibility with previously developed drivers, the soft interrupt and tasklet can completely replace the BH interface. Soft Interrupt is a set of static-defined lower-half interfaces with 32 interfaces that can be executed simultaneously on all processors, even if the two types are identical. Task is a lower half implementation mechanism with high flexibility and dynamic creation Based on Soft Interrupt. Two different types of tasklets can be executed simultaneously on different processors, but tasklets of the same type cannot be executed simultaneously. Tasklet is a product that seeks to balance performance and ease of use. Soft Interrupt must be statically registered during compilation, while tasklet can dynamically register through code. Now it's all about the 2.6 kernel. Let's talk about it. The linux2.6 Kernel provides three different forms of lower half implementation mechanisms: Soft Interrupt, tasklets, and work-on-column. These will be introduced in sequence. At this time, some people may think of the concept of the timer, the timer is indeed like this, but the timer provides a precise delay time, we will not do this here, so let's put it down first, we will talk about the timer later. Well, let's start with the detailed mechanisms:

1. Soft InterruptIn fact, there are not many soft interruptions, but there are many tasklets behind them, but the tasklet is implemented through soft interruptions. The Soft Interrupt code is located in/kernel/softirq. c. Soft interrupt is statically allocated during compilation, represented by the softirq_action structure, which is defined in linux/interrupt. h:

Struct softirq_action {void (* action) (struct softirq_action *); // void * data of the function to be executed; // parameter passed to the function };

Kernel/softirq. c defines an array containing 32 struct:

static struct softirq_action softirq_vec[32]

Each registered Soft Interrupt occupies one of the arrays, so a maximum of 32 soft interruptions may exist, which cannot be changed dynamically. Most drivers use tasklet to implement the lower half of them. Therefore, there are only six in the current kernel. In the above Soft Interrupt structure, the first item is the Soft Interrupt Processing Program. The prototype is as follows:

void softirq_handler(struct softirq_action *)

When the kernel runs a soft interrupt handler, it executes this action function. Its unique parameter is the pointer to the corresponding softirq_action struct. For example, if my_softirq points to the softirq_vec array implementation, the kernel will call the functions in the soft interrupt handler in the following way:

my_softirq->action(my_softirq)

A Soft Interrupt does not preempt another Soft Interrupt. In fact, the only thing that can preempt a Soft Interrupt is the interrupt processing program. However, other soft interruptions-or even the same type of soft interruptions-can be executed simultaneously on other types of machines. A registered Soft Interrupt can only be executed after it is marked-rasing the softirq is triggered. Generally, the interrupt handler will mark its soft interrupt before returning so that it can be executed later. Soft interruptions to be processed will be checked and executed in the following areas:

1. After processing a hardware interrupt 2. In the ksoftirqd kernel thread 3. In the code that displays the Soft Interrupt to be processed, such as the network subsystem.

The Soft Interrupt is executed in do_softirq (). If the Soft Interrupt is to be processed, do_softirq cyclically traverses each one and calls their processing program. The core part is as follows:

u32 pending = softirq_pending(cpu);if(pending){struct softirq_action *h = softirq_vec;softirq_pending(cpu) = 0;do{  if(pending &1)h->action(h);  h++;  pending >>=1;}while(pending);}

The above code checks and executes all the soft interrupts to be processed, softirq_pending (), and uses it to obtain the 32-Bit Bitmap of the Soft Interrupt to be processed ----- if the n-bit is set to 1, the n-bit Soft Interrupt waits for processing. Once the Soft Interrupt bitmap to be processed is saved, the actual Soft Interrupt bitmap can be cleared. Pending & 1 is to judge whether the first place of pending is set to 1. if pending is set to 0, no interruptions to be processed are displayed, because pending can be set to 32 bits at most, and a loop can only be executed to 32 bits at most. The Soft Interrupt is retained to the lower half of the system with the strictest and most important time requirements. Therefore, you should be clear before using it. The following describes how to use Soft Interrupt:

1. Allocate an index: during compilation, you must use an enumeration type created in <linux/interrupt. h> to declare the Soft interrupt statically. The kernel uses these indexes starting from 0 to indicate a relative priority, The smaller the index number, the more advanced the execution. Therefore, you can place your index number in a proper position based on your needs. 2. register the handler: Then, at runtime, call open_softirq () to register the interrupt handler, for example, the network subsystem, as shown below: open_softirq(NET_TX_SOFTIRQ,net_tx_action,NULL);open_softirq(NET_RX_SOFTIRQ,net_rx_action,NULL); The function has three parameters: Soft Interrupt index number, processing function and data field storage array. When a soft interrupt handler is executed, it allows the response to interrupt, but it cannot sleep. Run The Soft Interrupt of the current processor is disabled, but other processors can still execute other soft interrupts. In fact, if a Soft Interrupt is triggered again when it is executed Another processor can run its processing program at the same time. This means that strict lock protection is required for shared data. Most Soft Interrupt handlers process data by taking a single ticket (only belongs to one location Manager data) or some other techniques to avoid display locking, so as to provide better performance. 3. trigger Soft Interrupt: After the above two items, the new soft interrupt handler will be able to run. The raist_softirq (Interrupt index number) function can set a soft interrupt to a pending State so that it can be called next time. Run the do_softirq () function. This function must disable the interrupt before triggering a Soft Interrupt, and then return to the original state. If the interrupt is already disabled, you can call another function raise_softirq_irqoff (), this will bring some optimization results.

Triggering a soft interrupt in the interrupt handler is the most common form. The interrupt handler executes operations related to the hardware device, then triggers the Soft Interrupt, and finally exits. After the kernel executes the interrupt handler, it will immediately call the do_softirq () function. As a result, the Soft Interrupt begins to execute the interrupt processing program and leaves it to complete the remaining tasks.

2. Tasklets:Tasklet is implemented through Soft Interrupt, so they are also Soft Interrupt. It is represented by two types of soft interruptions: HI_SOFTIRQ and TASKLET_SOFTIRQ. The difference is that the former will be executed before the latter. Tasklets is represented by the tasklet_struct structure. Each struct represents a tasklet, which is defined in linux/interrupt. h:

struct tasklet_struct{struct tasklet_struct *next;unsigned long state;atomic_t count;void (*func)(unsigned long);unsigned long data;};

The func member in the struct is the processing program of tasklet, and data is its unique parameter. The value of state can only be 0, TASKLET_STATE_SCHED (indicating that the tasklet has been scheduled and is running) and TASKLET_STATE_RUN (indicating that the tasklet is running, it is used as an optimization only on a multi-processor system, and the single-processor system knows whether a single tasklet is running at any time. Count is the reference counter of tasklet. If it is not 0, tasklet is disabled and cannot be executed. tasklet is activated only when it is 0, this tasklet can be executed only when it is set to the suspended state. The scheduled tasklet (equivalent to the triggered Soft Interrupt) is stored in two single-processor data structures: tasklet_vec (Common tasklet) and task_hi_vec high-priority tasklet, both data structures are linked lists composed of tasklet_struct struct. Each tasklet_struct in the linked list represents a different tasklet. Tasklets is scheduled by tasklet_schedule () and tasklet_hi_schedule (). They receive a pointer to the tasklet_struct structure as a parameter. The two functions are very similar (the difference is that one uses TASKLET_SOFTIRQ and the other uses HI_SOFTIRQ). Operation details about tasklet_schedule:

1. Check whether the tasklet status is TASKLET_STATE_SCHED. If yes, the tasklet has been scheduled and the function returns. 2. Save the interruption status and disable local interruption. When executing the tasklet code, this ensures that the data on the processor will not be messed up. 3. Add the tasklet to the table header of the tasklet_vec linked list or task_hi_vec linked list of each processor. 4. Call the TASKLET_SOFTIRQ or HI_SOFTIRQ Soft Interrupt so that the tasklet is executed the next time do_softirq () is called. 5. Resume the interruption to the original state and return.

What have you done as the core tasklet_action () and tasklet_hi_action () of tasklet processing:

1. Disable interruption and search the tasklet_vec or tasklet_hi_vec linked list for the current processor. 2. Set the linked list on the current processor to NULL to clear the list. 3. The operation is interrupted accordingly. 4. cyclically traverse each tasklet to be processed on the linked list. 5. For a multi-processor system, check TASKLET_STATE_RUN to determine whether the tasklet is running on another processor. If it is running, do not execute it now. To the next tasklet to be processed. 6. If the current tasklet is not executed, set its status to TASKLETLET_STATE_RUN so that other processors will not execute it again. 7. Check whether the count value is 0 to ensure that the tasklet is not disabled. If tasklet is disabled, it is skipped to the next suspended tasklet. 8. Now we can confirm that this tasklet is not executed elsewhere and is set to the execution status so that it will not be executed in other parts and the reference counter is 0. Now we can execute Tasklet's processing the program. 9. Repeat the next tasklet until there are no remaining tasklets waiting for processing.

After talking about this, how should we use this tasklet? I have talked too much about the theory of driver in linux. But don't worry. I will continue to give a rough description of this topic for the sake of completeness:

1. Declare your tasklet: it can be set to static or dynamic, depending on whether you want to have a direct reference to tasklet or indirect reference. The static creation method (directly referenced) can use one of the following two macros (defined in linux/interrupt. h ):

DECLARE_TASKLET(name,func,data)DECLARE_TASKLET_DISABLED(name,func,data)

The difference between the two macros is that the reference counter has different initial values. The previous one sets the reference counter of the created tasklet to 0 to make it active, and the other one sets it to 1, in the forbidden status. The dynamic creation (indirect reference) method is as follows:

tasklet_init(t,tasklet_handler,dev);

2. compile the tasklet handler: The function type is void tasklet_handler (unsigned long data ). because it is implemented by Soft Interrupt, tasklet cannot sleep, that is, it cannot use semaphores or other blocking functions in tasklet. Because the tasklet operation allows the response to be interrupted, you must take precautions if some data is shared between the newly added tasklet and the interrupt handler. Two identical tasklets cannot be executed at the same time. If the newly added tasklet shares data with another tasklet or soft interrupt, you must implement proper lock protection.

3. Schedule Your Own tasklet: after the work on the front is finished, the scheduling is left. Implement the following method: tasklet_schedule (& my_tasklet). After the tasklet is scheduled, it can be run as long as there is a proper opportunity. If the same tasklet is scheduled again before it can run successfully, it will only run once. If the task has already started running, the new tasklet will be rescheduled and run again. One optimization strategy is that a tasklet is always executed on the processor that schedules it. Call tasklet_disable () to disable a specified tasklet. If the tasklet is currently being executed, this function will wait until it is executed and then return. It is also forbidden to call tasklet_disable_nosync (), but you do not have to wait for the tasklet to be executed before returning the result. This is not very secure (because it is impossible to estimate whether the tasklet is still being executed ). Tasklet_enable () activates a tasklet. You can use the tasklet_kill () function to remove a tasklet from the suspended pair column. This function waits for the tasklet to be executed and then moves it. Of course, nothing can prevent code elsewhere from rescheduling the tasklet. This function may cause sleep, so you are not allowed to use it in the interrupt context.

Next, I mentioned above that for Soft Interrupt, the kernel will choose a few special operations (usually when the interrupt handler returns ). Soft Interrupt is triggered frequently and may be triggered repeatedly. The result is that the process in the user space cannot obtain enough processor time because it is in a hunger state. At the same time, it is unacceptable to simply adopt policies that do not immediately handle soft interruptions that are triggered repeatedly. What are the two extreme but perfect situations:

1. As long as there is a soft interrupt that is triggered and waiting for processing, this execution will be responsible for handling it. The soft interrupt that is re-triggered will also be processed before this execution returns. The problem is that user processes may be ignored and In hunger. 2. Select the Soft Interrupt that is not re-triggered. When a Soft Interrupt is returned from the interrupt, the kernel will also check all pending soft interrupts and process them, but any soft interrupt that is manually re-triggered will They will not be processed immediately. They will be processed at the next Soft Interrupt execution time. The problem is that a new or re-triggered Soft Interrupt takes some time to be executed.

What I want now is to come up with a compromise. How good is that? The kernel developer really thought of it. The kernel selection scheme does not immediately handle soft interruptions that are re-triggered. As an improvement, when a large number of soft interruptions occur, the kernel will wake up a group of kernel threads to process these loads. These threads run at the lowest priority (nice value is 19 ). This solution ensures that the user program will not be able to deal with hunger due to lack of processing time when the soft interruption is heavy. Correspondingly, it can also ensure that the "excessive" Soft Interrupt will eventually be handled. Finally, in the idle system, this solution also performs well, and Soft Interrupt Processing is very fast (because the only memory thread will be scheduled immediately ). To ensure that there are idle processors, they will handle soft interruptions, so each processor is allocated with such a thread. All threads are named ksoftirad/n. The difference is that n corresponds to the processor number. Once the thread is initialized, it will execute an endless loop like the following:

for(;;){if(!softirq_pending(cpu))schedule();set_current_state(TASK_RUNNING);while(softirq_pending(cpu)){do_softirq();if(need_resched())schedule();}set_current_state(TASK_INTERRUPTIBLE);}

Softirq_pending () is responsible for detecting soft interruptions to be processed. After all the operations to be executed are completed, the kernel thread sets itself to the TASK_INTERRUPTIBLE state, arousing the scheduler to select other executable processes for running. Finally, as long as the do_softirq () function finds that the executed kernel thread has re-triggered itself, the Soft Interrupt kernel thread will be awakened.