The synchronization mechanism under Linux

2017-03-10

Recall that the original computer design, in the case of a single CPU, at the same time can only be a thread (under Linux for the process) to occupy the CPU, and the Linux kernel before 2.6 does not support kernel preemption, when the process is running in the system address, can interrupt the current operation of the only interrupt, and after the completion of the interrupt processing is found in the kernel, does not trigger the dispatch, only when the user space is returned to trigger the dispatch. Therefore, in the kernel of the shared resources in the case of a single CPU does not need to consider the synchronization mechanism, although the surface appears to be a number of processes at the same time, in fact, it is only the scheduler with a very small time granularity, scheduling the results of each process running, in fact, a pseudo-parallel. However, with the development of the Times, a single processor can not meet the demand for performance, multi-processor architecture has emerged. In this case, the work between multiple processors does not interfere with each other and can achieve true parallelism.

But there is only one operating system, many of which are globally shared, and even multiple CPUs cannot operate on their processes at the same time. However, in multi-processor cases, if we do not take precautions, it is very likely that the two processes simultaneously access the same variable, which can easily lead to the data out of sync. This situation is intolerable to both developers and users. In addition, kernel preemption is enabled in the kernel after 2.6, and even if the process is running in the system address space it is possible to preempt it, and the kernel synchronization mechanism is presented.

There are many synchronization mechanisms in the kernel, such as atomic operation, Semaphore, Spin lock, reader lock, RCU mechanism and so on. Each scenario has its advantages and disadvantages and is suitable for different scenarios.

Atomic operation

Atomic operations primarily protect a shared variable in the kernel, preventing the variable from being accessed concurrently causing data synchronization problems. To do this, a series of APIs are defined in the kernel that defines the atomic_t data type in the kernel, and its defined data operations are performed as a compilation instruction and are not interrupted in the middle. atomic_t defines a data type that is incompatible with the standard data type Int/short, the addition of the data cannot pass the standard operator, it must pass its own API, and here are some APIs for that type of operation

Static void atomic_add (int i, atomic_t * v)staticvoid atomic_sub (int i , atomic_t * v)staticint atomic_add_return (int i, atomic_t *v)  Staticlong atomic_sub_return (int i, atomic_t * v)

Other APIs are implemented based on the underlying API above, and are not listed here.

Signal Volume

Semaphores generally implement mutex operations, but you can specify the number of processes that are in the critical section, when the specified number is 1 o'clock, indicating that this is a mutex semaphore. The structure of the semaphore in the kernel is as follows

struct semaphore {    raw_spinlock_t        lock;     int         count;     struct list_head    wait_list;};

The start is a spin lock, which is used to protect the data structure, and count specifies the number of processes that the semaphore-associated resource allows simultaneous access to, and Wait_list is the list of processes waiting to access the resource. A benefit of semaphores, compared to spin locks, allows waiting for the process to sleep, rather than always polling the request. Therefore, the signal volume is more suitable for the longer critical region. Semaphore operation is very simple, initial initialization of a semaphore, before the critical resource requires a down operation to request a semaphore, execution completed the up operation to release resources.

The relevant code is as follows

void down (struct semaphore *sem) {    long  flags;    Raw_spin_lock_irqsave (&sem->lock, flags);     if 0 ) )        sem->count--;     Else         __down (SEM);    Raw_spin_unlock_irqrestore (&sem->lock, flags);}

void up (struct semaphore *sem) {    long  flags;    Raw_spin_lock_irqsave (&sem->lock, flags);     if (Likely (List_empty (&sem->wait_list)))        sem->count++;     Else         __up (SEM);    Raw_spin_unlock_irqrestore (&sem->lock, flags);}

For a down operation, the spin lock of the semaphore structure is obtained first, and the interrupt of the current CPU is closed, and if Count is also greater than 0, the resource is allocated directly, count--, otherwise the call down function blocks the current process, and the Down_common function is called directly in the down function.

StaticInlineint__sched __down_common (structSemaphore *sem,LongState ,LongTimeout) {    structTask_struct *task =Current ; structSemaphore_waiter Waiter; List_add_tail (&waiter.list, &sem->wait_list); Waiter.task=task; Waiter.up=false;  for (;;) {        if(Signal_pending_state (State, Task))Gotointerrupted; if(Unlikely (timeout <=0))            Gototimed_out;        __set_task_state (task, state); RAW_SPIN_UNLOCK_IRQ (&sem->Lock); Timeout=schedule_timeout (timeout); RAW_SPIN_LOCK_IRQ (&sem->Lock); if(waiter.up)return 0; } Timed_out:list_del (&waiter.list); return-etime; Interrupted:list_del (&waiter.list); return-eintr;}

First, a semaphore_waiter structure is constructed, which is inserted into the list of waiting processes in the semaphore structure. Timeout is a timeout that is not waiting for resources when set to less than or equal to 0. After these checks, the current process is set to the Task_interruptible state, which indicates a block that can be interrupted to wake up. The local interrupt is then turned on to indicate that the current task is over, and the following calls the Schedule_timeout process scheduler. After the specific switching process, the code in the lower half is executed the next time it is scheduled.

For the up operation, the spin lock is obtained first, and if the current waiting queue is empty, then simply increasing the count indicates that the available resources are increased, otherwise, the function implementation is relatively simple _up. First remove the corresponding node from the waiting list, set the structure's up signal to true, and then call the Wake_up_process function to wake the execution process. This wakes up, right? The process joins the Ready list and can be dispatched by the scheduler normally.

Static void __sched __up (struct semaphore *sem) {    struct semaphore_waiter *waiter = List_first_entry (&sem->wait_list,                        struct  semaphore_waiter, list);    List_del (&waiter->list);    Waitertrue;    Wake_up_process (task Waiter);}

Spin lock

The spin lock is probably the most widely used synchronization mechanism in the kernel, and it behaves as a two function in the kernel:

1, for data structure or variable protection

2, for the Protection of critical Section code

For the spin lock operation is very simple, its structure is spinlock_t, for the spin lock operation, according to the critical area of the not required level, there are a variety of APIs can choose

StaticInlinevoidSpin_lock (spinlock_t *Lock)StaticInlinevoidSpin_unlock (spinlock_t *Lock)StaticInlinevoidSPIN_LOCK_BH (spinlock_t *Lock)StaticInlinevoidSPIN_UNLOCK_BH (spinlock_t *Lock)StaticInlinevoidSPIN_LOCK_IRQ (spinlock_t *Lock)StaticInlinevoidSPIN_UNLOCK_IRQ (spinlock_t *Lock)

The first and most basic is the spin_lock, used to obtain the spin lock, before the specific acquisition will call preempt_disable prohibit kernel preemption, so the spin lock protection of the critical code during execution will not be dispatched. The nature of this council's critical code can be called SPIN_LOCK_BH to prohibit soft interrupts or by calling Spin_lock_irq to prohibit interrupts on the local CPU. The code that has the spin lock protection cannot go to sleep because the CPU waiting to acquire the lock will always poll, do nothing else, and if you sleep in the critical area, it consumes more CPU performance.

The above function acquires the lock and release lock primarily for the protection of the critical code, and the operation itself is an atomic operation.

For the protection of data structures, spin locks are often embedded as a field in the data structure, before the operation of the specific structure, the need to obtain a lock, operation completed release lock.

Reader-writer lock

Reader's problem is actually for read and write operations to do the processing, you can see the other synchronization mechanism does not distinguish between read and write operations, as long as the thread access, you need to lock, but a lot of resources in the case is not a write operation, can allow multi-process access. Therefore, in order to improve efficiency, reader-writer Lock was born. Read-write lock in the write operation, need to add writelock, at this time only one thread can enter the critical section, while performing a read operation, add Readlock, at this time can allow multiple threads to enter the critical section. It is suitable for critical sections where read operations are significantly more than write operations.

RCU mechanism

The RCU mechanism is a newer kernel synchronization mechanism that provides two types of protection: on data structures and on linked lists. Applied fairly frequently in the kernel.

RCU mechanism conditions of use:

• Access to shared resources is most of the time read-only, with relatively few write operations.
• Within the range of code protected by RCU, sleep cannot be entered.
• Protected resources must be accessed through pointers.

RCU protects the data structure, cannot dereference its pointer, that is, cannot *ptr obtain its content, must use its corresponding API. The code that also deserializes the pointer and uses its result must be protected with Rcu_read_lock () and Rcu_read_unlock ().

If you want to modify the object that PTR points to, you need to first create a copy and then call Rcu_assign_pointer (PTR,NEW_PTR) to modify it. So in this case, the protected data structure allows read and write concurrent execution, because essentially manipulating two structures, the pointer is only modified after the old data structure has been accessed.

Memory and optimization barriers

When looking at the kernel source code, often see the appearance of barrier (), the equivalent of a wall, so that the compiler before the end of the barrier before processing the code, will not deal with the code behind the barrier. Originally in order to improve the efficiency of code execution, the compiler will be appropriate to the code to rearrange the instructions, in general, this rearrangement does not affect the program function, but the compiler is not a person, some of the order of strict requirements of the code, it is likely not to be accurately recognized by the compiler, such as shutdown and enable preemption code, so, If the compiler moves the core code out of the off-preemption interval, it is likely to affect the final result, so it is time to add a memory barrier after the preemption is closed, guaranteeing that no subsequent code will be queued.

The synchronization mechanism under Linux