Mutual exclusion in the kernel

Article title: mutual exclusion in the kernel. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.
/* E4gle, however, I think I should leave something and have no time to write it. I occasionally saw this brother's article and thought it was exactly what I wanted to organize. so I would like to share it with you, concerning the bottom_half and interruption issues, you must never perform file read/write operations on the TCP/IP half-bottom. otherwise, panic will be used. this is exactly what I have to do with the enhancements in linux, I 've been depressed for a long time. welcome to discuss it.
*/
Mutual exclusion in the kernel
By wheelz
After reading the previous discussions, I have some ideas to discuss with you.
It should be clarified that the choice of mutex is not based on the size of the critical section, but on the nature of the critical section and the code, that is, the kernel execution path.
Strictly speaking, semaphore and spinlock_XXX are mutually exclusive methods at different levels. the implementation of the former depends on the latter. this is a bit like the relationship between HTTP and TCP, both of which are protocols, but the layers are different.
First, semaphore is process-level and used for resource mutex among multiple processes. although it is also in the kernel, the kernel execution path is the identity of the process, it represents a process to compete for resources. If the competition fails, there will be a context switch, the process can go to sleep, but the CPU will not stop, and other execution paths will be run. In terms of concept, this is not directly related to a single CPU or multiple CPUs, but only in the implementation of semaphore itself, in order to ensure the atomicity of semaphore structure access, in multiple CPUs, a spinlock is required for mutual exclusion.
In the kernel, data access from each execution path of the kernel must be mutually exclusive. this is the most basic problem of mutual exclusion, that is, to maintain the atomicity of data modification. The implementation of semaphore also depends on this. In a single CPU, the main problems are the interruption and bottom_half. Therefore, the switch can be interrupted. In the multi-CPU mode, other CPU interference is added, so you need to use a spinlock to help. These two parts are combined to form a spinlock_XXX. It features that, once the CPU enters the spinlock_XXX, it will not do anything else, but will remain empty until the lock is successful. Therefore, this determines that the critical section locked by the spinlock_XXX cannot be stopped, nor context switch. after the data is accessed, it is necessary to get out quickly so that other execution paths in the idling can obtain the spinlock. This is also the principle of the spinlock. If context switch is required for the current execution path, it is necessary to release the spinlock before schedule (). Otherwise, it is prone to deadlock. Because there is no context in the interrupt and bh, the context switch cannot be implemented. you can only wait for the spinlock by idling. if your context switch is gone, who knows that the monkey will be back in the year of the horse and month of the monkey.
Because the intention and purpose of spinlock is to ensure the atomicity of data modification, there is no reason to stay in the critical section of the spinlock.
Spinlock_XXX has many forms, including
Spin_lock ()/spin_unlock (),
Spin_lock_irq ()/spin_unlock_irq (),
Spin_lock_irqsave/spin_unlock_irqrestore ()
Spin_lock_bh ()/spin_unlock_bh ()
Local_irq_disable/local_irq_enable
Local_bh_disable/local_bh_enable
So under what circumstances should we use it? It depends on the kernel execution path and the kernel execution path. We know that the main execution paths in the kernel are:
1. there is a process context in the kernel state of the user process, which mainly indicates that the process is executing system calls.
2. interrupted, abnormal, or self-trapped. in terms of concept, there is no process context and it cannot be performed.
Context switch.
3 bottom_half. in terms of concept, there is no process context at this time.
4 at the same time, the same execution path may run on other CPUs.
In this way, considering these four factors, the data that we want to mutually exclusive will be identified by these four factors.
Which of the following statements can be used to access the system. If it is mutually exclusive with other CPUs, you need to use spin_lock/spin_unlock. if it is mutually exclusive with irq and other CPUs, you need to use
Spin_lock_irq/spin_unlock_irq: if it is mutually exclusive with irq and other CPUs and saves EFLAG status, it is necessary to use spin_lock_irqsave/spin_unlock_irqrestore, you need to use spin_lock_bh/spin_unlock_bh. if you do not need to be mutually exclusive with other CPUs, use local_irq_disable/local_irq_enable as long as you and irq are mutually exclusive,
If you do not need to be mutually exclusive with other CPUs, use local_bh_disable/local_bh_enable as long as you and bh are mutually exclusive,
And so on. It is worth noting that the mutex on the same data is in different kernel execution paths,
The format may be different (see the example below ).
For example. In the interrupt section, there is an irq_desc_t type structure array variable irq_desc [],
Each member of the array corresponds to an irq description structure, which contains the irq response function.
There is a spinlock in the irq_desc_t structure to ensure the mutex of access (modification.
For an irq member, irq_desc [irq] has two paths for accessing the kernel. one is
Setup_irq usually occurs during module initialization, or
The initialization phase of the system; the second is in the interrupt response function (do_IRQ ). The code is as follows:
Int setup_irq (unsigned int irq, struct irqaction * new)
{
Int shared = 0;
Unsigned long flags;
Struct irqaction * old, ** p;
Irq_desc_t * desc = irq_desc + irq;
/*
* Some drivers like serial. c use request_irq () heavily,
* So we have to be careful not to interfere with
* Running system.
*/
If (new-> flags & SA_SAMPLE_RANDOM ){
/*
* This function might sleep, we want to call it first,
* Outside of the atomic block.
* Yes, this might clear the entropy pool if the wrong
* Driver is attempted to be loaded, without actually
* Installing a new handler, but is this really a problem,
* Only the sysadmin is able to do this.
*/
Rand_initialize_irq (irq );
}
/*
* The following block of code has to be executed atomically
*/
[1] spin_lock_irqsave (& desc-> lock, flags );
P = & desc-> action;
If (old = * p )! = NULL ){
/* Can't share interrupts unless both agree */
If (! (Old-> flags & new-> flags & SA_SHIRQ )){
[2] spin_unlock_irqrestore (& desc-> lock, flags );
Return-EBUSY;
}
/* Add new interrupt at end of irq queue */
Do {
P = & old-> next;
Old = * p;
} While (old );
Shared = 1;
}
* P = new;
If (! Shared ){
Desc-> depth = 0;
Desc-> status & = ~ (IRQ_DISABLED | IRQ_AUTODETECT | IRQ_WAITING );
Desc-> handler-> startup (irq );
}
[3] spin_unlock_irqrestore (& desc-> lock, flags );
Register_irq_proc (irq );
Return 0;
}
Asmlinkage unsigned int do_IRQ (struct pt_regs regs)
{
/*
* We ack quickly, we don't want the irq controller
* Thinking we're re snobs just because some other CPU has
* Disabled global interrupts (we have already done
* INT_ACK cycles, it's too late to try to pretend to
* Controller that we aren't taking the interrupt ).
*
* 0 return value means that this irq is already being
* Handled by some other CPU. (or is disabled)
*/
Int irq = regs. orig_eax & 0xff;/* high bits used in ret_from _ code */
Int cpu = smp_processor_id ();
Irq_desc_t * desc = irq_desc + irq;
Struct irqaction * action;
Unsigned int status;
Kstat. irqs [cpu] [irq] ++;
[4] spin_lock (& desc-> lock );
Desc-> handler-> ack (irq );
/*
REPLAY is when Linux resends an IRQ that was dropped earlier
WAITING is used by probe to mark irqs that are being tested
*/
Status = desc-> status &~ (IRQ_REPLAY | IRQ_WAITING );
Status | = IRQ_PENDING;/* we _ want _ to handle it */
/*
* If the IRQ is disabled for whatever reason, we cannot
* Use the action we have.
*/
Action = NULL;
If (! (Status & (IRQ_DISABLED | IRQ_INPROGRESS ))){
Action = desc-> action;
Status & = ~ IRQ_PENDING;/* we commit to handling */
Status | = IRQ_INPROGRESS;/* we are handling it */
}
Desc-> status = status;
/*
* If there is no IRQ handler or it was disabled, exit early.
Since we set PENDING, if another processor is handling
A different instance of this same irq, the other processor
Will take care of it.
*/
If (! Action)
Goto out;
/*
* Edge triggered interrupts need to remember
* Pending events.
* This applies to any hw interrupts that allow a second
* Instance of the same irq to arrive while we are in do_IRQ
* Or in the handler. But the code here only handles the _ second _
* Instance of the irq, not the third or fourth. So it is mostly
* Useful for irq hardware that does not mask cleanly in
* SMP environment.
*/
For (;;){
[5] spin_unlock (& desc-> lock );
Handle_IRQ_event (irq, & regs, action );
[6] spin_lock (& desc-> lock)