Linux Interrupt Context

First, preface

Every Linux driver is aware of the criteria for not sleeping in the context of an interrupt. But why can't the kernel API that causes sleep be called in interrupt context? What would happen if the driver did this? This is the topic discussed in this article. In order to understand this topic, we have designed some very simple programs of driver and user space, actually do experiments to observe the experimental results, and finally give the results and analysis.

BTW, this experiment is done on the X86 64bit + standard 4.4 kernel.

Second, the test procedure

1. CST Drive Module

We first prepare a driver that can sleep in an interrupt context, where I call the context Schedule test module (later referred to as the CST module). This driver is similar to a "troublemaker" lurking in the kernel, randomly hitting a process every 1 seconds, and then raising the schedule. First, prepare a makefile with the following code:

Kernelsrc? =/home/xxxx/work/linux-4.4.6

Default
$ (make)-C $ (KERNELSRC) m=$ $PWD

Clean
$ (make)-C $ (KERNELSRC) m=$ $PWD Clean

Supposedly the code of XXXX should be my name, if you are willing to test in your environment, you can change to your name, of course, the most important thing is to need a version of the kernel code. In the kernel upgrade documentation, I have compiled the kernel in the/home/xxxx/work/linux-4.4.6 directory and upgraded My computer to the 4.4.6 kernel, and if you would like to be able to upgrade according to that document, there might be some version issues to deal with. In addition to makefile, you need a Kbuild file:

Obj-m: = CST.O

Of course, the most important is the source code of the CST module:

#include
#include
#include
#include

#define DRIVER_DESC "Context Schedule test DRIVER"

static struct timer_list Cst_timer;

static void Cst_timer_handler (unsigned Long data)
{
struct Task_struct *p = current;

Pr_info ("=====in timer handler=====\n");
Pr_info ("CST shoot%16s [%x] task:\n", P->comm, Preempt_count ());
Mod_timer (&cst_timer, jiffies + HZ);
Schedule ();
}

static int __init cst_init (void)
{
Init_timer (&cst_timer);
Cst_timer.function = Cst_timer_handler;
Cst_timer.expires = jiffies + HZ;
Add_timer (&cst_timer);

Pr_info (driver_desc ": Init on Cpu:%d\n", smp_processor_id ());
return 0;
}
Module_init (Cst_init);

static void __exit cst_exit (void)
{
Del_timer_sync (&cst_timer);
Pr_info (driver_desc ": exit\n");
}
Module_exit (Cst_exit);

Module_description (DRIVER_DESC);
Module_author ("Linuxer <[email protected]>");
Module_license ("GPL");

The code is very simple, needless to say, direct make can compile the Cst.ko kernel module.

2. User Space test Program

For more convenient testing, we need to prepare a "victim" with the following code:

#include
#include

int main (int argc, char **argv)
{
int i = 0;

while (1) {
sqrt (rand ());

if ((i% 0xffffff) = = 0)
printf ("=\n");

if ((i% 0xfffffff) = = 0)
printf ("Haha......still alive\n");

i++;
}

return 0;
}

This code is also very simple: the constant generation of a random number, and the operation of its square root, at the time of making, to the user output some characters, indicating their state. When the program executes, most of the time in the user state (operation), occasionally into the kernel State (printf). This process does not know that in the kernel state has a CST module, every second to launch a dormant arrow, may hit the user state, may also hit the kernel state, see luck, but anyway, the process will be shot after the sleep.

Iii. performing tests and observing results

1, first the user space test program run up

To test the missile (hehe, our CST module is a troublemaker) performance, must have target drone or target ship. Of course, you can not use the "target drone" program, but the troublemaker CST always hit the swapper process, a bit uninteresting, so here we need to put our user space of the test program run up, so that the CPU first active up.

It is important to note that on multicore CPUs, we need to run a few more "target drone" processes to keep the system from always entering the idle state. For example, my T450 is a 4-core CPU, so I need to run 4 target drone programs to get the 4 CPU cores in the system to dry up. This can be confirmed by the following command:

Ps–eo COMM,PSR | grep CST

BTW, the target drone program is cst_test. With the above command, you can see that the system is running four cst_test processes, on 4 CPUs respectively.

2. Inserting the kernel module

Target drone is ready, it's time to launch a trick, the command is as follows:

sudo insmod./cst.ko

Once the CST kernel module is plugged in, the troublemaker begins to operate, firing every 1 seconds, always having a hapless egg hit and dispatched. Of course, in our tests, it is always cst_test that the process is hit.

3. Observation results

When everything is ready, it's time to take a little bench and sit down and watch the show. Of course, we need an observation tool and enter the following command:

sudo tail–f/var/log/messages

In the CST module above, the output is not directly to the console, so we need to see the CST running through the kernel log.

Iv. Results and analysis

1. Results

Very strange, everything is normal, the system is not dead, CST module is also running normally, cst_test process also always maintain alive state, constantly running in the boring square root, printing a boring string. The only exception is the log, which dumps the stack once every 1 seconds.

2. Analysis

When the CST module hits the cst_test process, either userspace or kernel space, the context in which the interrupt occurs is saved on the kernel stack, the only difference being that if it happens in userspace, the kernel stack is empty at the moment the interrupt occurs, And if in kernel space, the kernel stack already has cst_test through the system calls into the kernel of the field and the system calls the various functions of the stack frame, when the interruption occurs, on the stack top of the current kernel stack continues to press into the interrupt scene, Then is the stack frame of each function that interrupts processing, and finally the stack frame of cst_timer_handler, because the schedule function is called, the field of the cst_test process is continued to press into the kernel stack, and the scheduler decides the next scheduled process.

Although the cst_test process is scheduled, but still in the Runqueue, the scheduler will be in the appropriate time to dispatch cst_test process re-enter the execution state, when the resumption of its execution is OK, CPU execution cst_timer_ After the handler function schedule the code, the execution of the unfinished interrupt context continues, and the site is resumed from the kernel stack, and everything is returned in the original way.

Of course, the tests here seem to be OK, but that's not to say that it's free to call the kernel API that causes sleep in the context of the interrupt, because here's a simple example that could actually lead to system deadlock. For example, the kernel state holds the lock when it is interrupted, and then the dispatch occurs. Interested students can modify the above code to experiment with this situation.

3. Why

Finally, I return to this specific technical question: Why can't I call the kernel API that causes sleep in interrupt context?

My view is this: The scheduler is a prerequisite for every OS, and before the coding phase, we often have to develop our design concepts. For the Linux scheduler, its goal is to dispatch a thread, and a thread is the dispatch entity (not considering group Sched). Is the interrupt context a dispatch entity? Of course not, it does not have a dedicated task struct, the kernel does not dispatch. This is the decision of the Scheduler designer, which makes the scheduler simple and beautiful to design.

Based on the above design concept, the interrupt context (hard IRQ and SOFTIRQ context) does not participate in scheduling (without considering the disconnection threaded), which is the processing mechanism of the asynchronous event, and the goal is to complete the processing as soon as possible and return to the scene. Therefore, the priority of all interrupt contexts is higher than the process context, that is, for the user process (either kernel or user) or kernel thread, unless the CPU is disable local interrupt, if the interrupt occurs, They are not capable of blocking the execution of the interrupt context preemption of the current process context.

As a result, the designers of Linux kernel have established rules:

1. Interrupt context is not a dispatch entity

2, the interrupt context priority is higher than the process context

Scheduling in an interrupt context will undoubtedly break the rules, so you cannot call the blocking function in a hard interrupt, soft interrupt environment. However, at the time of the implementation of the Linux scheduler, it was detected that dispatching the schedule function in the context of the interrupt did not force Linux into the panic, perhaps the Linux developers think that a good kernel scheduler in any case also do their best to let the system run down. However, in the vendor's own kernel, often modify the scheduler behavior, in the context of the interrupt detection of the dispatch is directly panic, for the kernel developers, so better, can find the problem as early as possible.

Linux Interrupt Context