"Linux kernel design and implementation" reading notes (10)-Kernel synchronization method

The kernel provides several ways to prevent competitive conditions, and understanding the usage scenarios of these methods helps us to choose the appropriate synchronization method when writing kernel code.

This ensures the security of critical sections in your code, while minimizing the loss of performance.

Main content:

• Atomic operation
• Spin lock
• Read/write Spin lock
• Signal Volume
• Read/write signal volume
• Mutex
• Completion variables
• Large kernel Lock
• Sequential lock
• Prohibit preemption
• Order and barrier
• Summarize

1. Atomic operation

Atomic operations are guaranteed by the compiler to ensure that a thread's operations on the data are not interrupted by other threads.

There are 2 types of atomic operations:

1. Atomic integer operations, with 32-bit and 64-bit. Header files are <asm/atomic.h> and <asm/atomic64.h> respectively
2. Atomic bit operation. Header Files <asm/bitops.h>

The atomic Operation API is simple, see the corresponding header file.

The atomic operation header file is related to the specific architecture, such as the associated header file for the x86 architecture in Arch/x86/include/asm/*.h

2. Spin Lock

Atomic operation can only be used in the case of only one variable in the critical section, in practical application, the situation of the critical section is much more complicated.

For complex critical zones, there are several synchronization methods available in the Linux kernel, and spin locks are one of them.

The spin lock feature is that when a thread acquires a lock, other threads trying to acquire the lock are looping around to acquire the lock until the lock is re-usable.

Since threads actually get this lock in a loop, it can be a waste of CPU processing time, so it's best to use a spin lock for critical sections that can be processed very quickly.

The implementation of the spin lock is related to architecture, so the corresponding header file <asm/spinlock.h> is in the code of the relevant architecture.

There are 2 points to note when using the spin lock:

1. Spin locks are not recursive, and recursive requests for the same spin lock will Self-lock themselves.
2. The interrupt on the current processor is suppressed before the thread acquires the spin lock. (prevents the thread that acquires the lock and interrupts form a race condition)
   For example: After the current thread acquires a spin lock, the interrupt handler is interrupted in the critical section, and the interrupt handler acquires the lock exactly as well.
   The interrupt handler waits for the current thread to release the lock, and the current thread waits for the code to execute the critical section and release the lock after the interrupt has finished executing.

The use of a spin lock in the operation of the lower half of the interrupt handling is especially necessary for caution:

1. The next half of processing and the process context share the data, because the processing of the lower part can preempt the process context code,
   So the process context disables the execution of the lower part before locking the shared data, allowing the lower half to be executed when unlocked.
2. When the interrupt handler (top half) and the lower half process the shared data, the interrupt processing (top half) can preempt the lower half of the execution,
   So the lower half disables interrupt processing (the top half) before locking the shared data, allowing the execution of interrupts to be unlocked.
3. The same tasklet cannot run at the same time, so shared data in the same tasklet does not need to be protected.
4. When sharing data in different classes of tasklet, one of the Tasklet obtains the lock without prohibiting the execution of other tasklet because there is no tasklet preemption on the same processor
5. Soft interrupts of the same type or non-identical type when sharing data, do not prohibit the lower half, because there is no soft interrupt on the same processor to preempt each other situation

The spin lock method list is as follows:

Method	Describe
Spin_lock ()	Gets the specified spin lock
SPIN_LOCK_IRQ ()	Disables local interrupts and acquires the specified lock
Spin_lock_irqsave ()	Saves the current state of the local interrupt, disables local interrupts, and obtains the specified lock
Spin_unlock ()	Releases the specified lock
SPIN_UNLOCK_IRQ ()	Releases the specified lock and activates the local interrupt
Spin_unlock_irqstore ()	Releases the specified lock and allows local interrupts to revert to the previous state
Spin_lock_init ()	Dynamic initialization of the specified spinlock_t
Spin_trylock ()	An attempt was made to get the specified lock, or 0 if not obtained
Spin_is_locked ()	Returns non 0 if the specified lock is currently being fetched, otherwise 0

3. Read/write Spin lock

1. In addition to the spin lock, the read-write spin lock has the following characteristics, as well as the normal spin locks:
   Read locks are shared between
   That is, after a thread holds a read lock, other threads can also hold the lock in a read manner.
2. Write locks are mutually exclusive.
   That is, after a thread holds a write lock, other threads cannot hold the lock in a read or write manner.
3. The read-write lock is mutually exclusive
   That is, after a thread holds a read lock, the other thread cannot hold the lock in a written manner.

Note : Read and write locks should be used separately, can not be mixed use, otherwise it will cause deadlock.

The normal use method:

Define_rwlock (Mr_rwlock); Read_lock (&mr_rwlock);/* critical section (read only) .... */read_unlock (&mr_rwlock); Write_lock (& Mr_lock);/* critical section (read/write) ... */write_unlock (&mr_lock);

When mixed use:

/* Get a read lock */read_lock (&mr_lock);/* When acquiring a write lock, because the read-write lock is mutually exclusive, * so the write lock will always spin waiting for the read lock to be released, while the read lock is also waiting for the write lock to complete after the following code continues. * Thus causing the read-write lock to wait for each other, forming a deadlock. */write_lock (&mr_lock);

Read/write lock related files refer to <asm/rwlock.h> in each architecture

The correlation functions for read and write locks are as follows:

Method	Describe
Read_lock ()	Gets the specified read lock
READ_LOCK_IRQ ()	Disable local interrupts and obtain a specified read lock
Read_lock_irqsave ()	Stores the current state of local interrupts, disables local interrupts, and obtains a specified read lock
Read_unlock ()	Releases the specified read lock
READ_UNLOCK_IRQ ()	Releases the specified read lock and activates the local interrupt
Read_unlock_irqrestore ()	Releases the specified read lock and restores the local interrupt to the pre-specified state
Write_lock ()	Get the specified write lock
WRITE_LOCK_IRQ ()	Disable local interrupts and obtain a specified write lock
Write_lock_irqsave ()	Stores the current state of local interrupts, disables local interrupts, and obtains a specified write lock
Write_unlock ()	Releases the specified write lock
WRITE_UNLOCK_IRQ ()	Releases the specified write lock and activates the local interrupt
Write_unlock_irqrestore ()	Releases the specified write lock and restores the local interrupt to the pre-specified state
Write_trylock ()	An attempt was made to obtain a specified write lock, and a non-0 value is returned if the write lock is not available
Rwlock_init ()	Initializes the specified rwlock_t

4. Signal Volume

The semaphore is also a kind of lock, and the spin lock is different, when the thread does not get the semaphore, it does not loop like a spin lock to try to acquire the lock,

Instead, it goes to sleep until a semaphore is released, waking the thread of sleep and entering the critical section.

Because the thread sleeps when the semaphore is used, the wait process does not consume CPU time. Therefore, the semaphore is suitable for the critical section with long waiting time.

The place where the semaphore consumes CPU time is to make the thread sleep and wake the thread,

If the CPU time (which causes the thread to sleep + wake-up thread) > thread spins the CPU time to wait, then consider using a spin lock.

The semaphore has two value semaphore and counting signal Quantity 2 kinds, wherein two value signal quantity is more commonly used.

A two-value semaphore represents a semaphore with only 2 values, which is 0 and 1. A semaphore of 1 o'clock indicates that the critical section is available and the semaphore is 0 o'clock, indicating that the critical section is inaccessible.

The binary semaphore surface looks similar to the spin lock, except that the thread that is competing for the spin lock will always try to get the spin lock,

A thread that is contending for semaphores will go to sleep when the semaphore is 0 o'clock, and then wake up when the semaphore is available.

The Count Semaphore has a count value, such as a count of 5, indicating that there can be 5 threads accessing the critical section at the same time.

Semaphore correlation Function Reference: <linux/semaphore.h> Implementation method Reference: KERNEL/SEMAPHORE.C

Here's how to use the semaphore:

/* Define and declare a semaphore, named Mr_sem, for semaphore Count */static Declare_mutex (MR_SEM);/* Try to get semaphore .... When the signal does not get successful, go to sleep * At this point, the thread status is Task_ Interruptible */down_interruptible (&MR_SEM);/* can also be used here: * Down (&MR_SEM); * This method puts the thread state to task_uninterruptible after the sleep *//* critical section ... *//* release a given semaphore */up (&MR_SEM);

The more commonly used is the down_interruptible () method, because sleep in task_uninterruptible mode cannot be awakened by a signal.

For task_interruptible and task_uninterruptible add a note:

• Task_interruptible-Can interrupt sleep, accept the signal and be awakened, or be explicitly awakened (such as the WAKE_UP () function) after all wait conditions have been reached.
• Task_uninterruptible-cannot interrupt sleep, but is explicitly awakened (such as the WAKE_UP () function) only after the waiting conditions have been fully reached.

The semaphore method is as follows:

Method	Describe
Sema_init (struct semaphore *, int)	Initializes dynamically created semaphores with the specified count value
Init_mutex (struct semaphore *)	Initializes dynamically created semaphores with a count value of 1
init_mutex_locked (struct semaphore *)	Initializes dynamically created semaphores with a count value of 0 (initially locking state)
down_interruptible (struct semaphore *)	To attempt to get the specified semaphore, or enter an interruptible sleep state if the semaphore is already being contended
Down (struct semaphore *)	To attempt to obtain the specified semaphore, or enter a non-interruptible sleep state if the semaphore is already being contended
Down_trylock (struct semaphore *)	To attempt to obtain the specified semaphore, and immediately returns a non-0 value if the semaphore has been contended
Up (struct semaphore *)	To release the specified semaphore, or wake up one of the tasks if the sleep queue is not empty

The semaphore structure is specified as follows:

/* Please don ' t access all members of this structure directly */struct semaphore {    spinlock_t        lock;    unsigned int        count;    struct List_head    wait_list;};

It can be found that there is a spin lock in the semaphore structure, the role of this spin lock is to ensure that the signal volume down and up operations will not be interrupted by interrupt handlers.

5. Read and write signal volume

The relationship between read and write semaphores and semaphores is similar to that between read and write spin locks and normal spin locks.

Read-write Semaphore is a two-value signal, that is, the maximum count of 1, increase the reader, the counter is unchanged, increase the writer, the counter is reduced by one.

That is to say, read and write semaphore protection of the critical area, at most only one writer, but can have multiple readers.

Read and write the relevant contents of the semaphore see:<asm/rwsem.h> the specific implementation is related to the hardware architecture.

6. Mutex

A mutex is also a sleep-capable lock, equivalent to a two-value semaphore, which simply provides a simpler API and uses a more restrictive scenario, as shown below:

1. The count value of a mutex can only be 1, which means that only one thread is allowed to access the critical section
2. Locking and unlocking in the same context
3. No recursive locking and unlocking
4. Process cannot exit when holding a mutex
5. Mutexes cannot be used in interrupts or the lower half, i.e. mutexes can only be used in the context of a process
6. A mutex can only be managed by the official API and cannot be manipulated by writing its own code.

In the face of the mutex and the selection of semaphores , as long as the use of the mutex satisfies the mutex as much as possible.

In the face of the mutex and the choice of spin lock , see the following table:

Demand	Recommended Lock-up method
Low Overhead plus lock	Priority use of spin lock
Short-term lock	Priority use of spin lock
Long-term lock	Preferential use of mutexes
Lock in interrupt context	Use spin lock
Holding a lock requires sleep	Using mutexes

Mutex header file:<linux/mutex.h>

Common mutex methods are as follows:

Method	Describe
Mutex_lock (struct Mutex *)	Locks the specified mutex and sleeps if the lock is not available
Mutex_unlock (struct Mutex *)	Unlocks the specified mutex
Mutex_trylock (struct Mutex *)	An attempt was made to get the specified mutex, or 1 if successful, otherwise the lock is fetched and 0 is returned.
mutex_is_locked (struct Mutex *)	Returns 1 if the lock is already contention, otherwise 0

7. Complete the variable

The mechanism for completing a variable is similar to the semaphore,

For example, after a thread a enters the critical section, another thread B waits on the completion variable, thread A finishes the task out of the critical section, and uses the completion variable to wake up thread B.

Complete the header file for the variable:<linux/completion.h>

The API to complete the variables is also simple:

Method	Describe
Init_completion (struct completion *)	Initializes the specified dynamically created completion variable
Wait_for_completion (struct completion *)	Waits for the specified completion variable to accept the signal
Complete (struct completion *)	Signaled to wake up any waiting tasks

Examples of using the completion variables can be found in: KERNEL/SCHED.C and KERNEL/FORK.C

It is generally possible to consider using completion variables when 2 tasks require simple synchronization.

8. Large Kernel Lock

The large kernel lock is no longer used, and only exists with some legacy code.

9. Sequential Lock

Sequential Lock provides a simple implementation mechanism for reading and writing shared data.

The previously mentioned read-write spin lock and read/write signal volume, after the read lock is obtained, the write lock can not be obtained again,

In other words, you must wait for all read locks to be released before you can write to the critical section.

Sequential locks are different, and write locks can still be acquired in the case of read locks.

Read operations that use sequential locks check sequential lock sequence values before and after reading, indicating that a write operation occurs during the read process, if the value does not match.

The read operation is executed again until the sequence values before and after the read are the same.

do{/    * Read the sequence value of the Order lock foo before *    /seq = Read_seqbegin (&foo); ...} while (Read_seqretry (&foo, seq));/* Sequential lock Foo Returns True when the sequence value!=seq and vice versa */

Sequential lock first guarantees that the write lock is available, so it is suitable for those who have a lot of readers, who write very little, and write better than read scenes.

Examples of use of sequential locks can be found in: Kernel/timer.c and KERNEL/TIME/TICK-COMMON.C files

10. No preemption

In fact, the use of spin lock can prevent the kernel preemption, but sometimes only need to prohibit kernel preemption, do not need to be like a spin lock even if the interrupt is blocked.

This is the time to use a method that prohibits kernel preemption:

Method	Describe
Preempt_disable ()	Increase preemption count value, thereby preventing kernel preemption
Preempt_enable ()	Reduce preemption calculations and when the value is reduced to 0 o'clock check and execute the scheduled tasks that are pending
Preempt_enable_no_resched ()	Activates kernel preemption but no longer checks for any suspended tasks that need to be scheduled
Preempt_count ()	Return preemption Count

The preempt_disable () and preempt_enable () can be nested, and the number of disable and enable should eventually be the same.

Prevent preemption of header files See:<linux/preempt.h>

11. Order and Barriers

For a piece of code, the compiler or processor may perform some optimizations on the execution order as it is compiled and executed, which makes the execution order of the code somewhat different from the code we write.

In general, this is not a problem, but under concurrency conditions, there may be situations where the values obtained are inconsistent with expectations

For example, the following code:

/  * * Thread A and thread B share variables A and b * Initial values a=1, b=2 */int a = 1, b = 2;/* * Assume thread A and B operations */void thread_a () {    a = 5;    b = 4;} /* * Assume that the operations of A and B in thread B */void thread_b () {    if (B = = 4)        printf ("a =%d\n", a);}

Because of optimizations in the compiler or processor, the order of assignment in thread A may be assigned by B before a value is assigned.

So if thread A is b=4; The execution is complete, a=5; Before execution, thread B begins execution, and thread B prints the initial value of a of 1.

This is inconsistent with what we expected, we expect a to be assigned before B, so thread B either does not print the content, and if it prints, the value of a should be 5.

In some cases of concurrency, a series of barrier methods are introduced to prevent compiler and processor optimizations in order to ensure code execution.

Method	Describe
RMB ()	Prevents a reorder of loading actions that span barriers
Read_barrier_depends ()	Prevent loading actions with data dependencies that span barriers from reordering
WMB ()	Prevent reordering of storage actions that span barriers
Obj	Prevents loading and storage actions across barriers from reordering
SMP_RMB ()	RMB () function on SMP with barrier () function on up
Smp_read_barrier_depends ()	Read_barrier_depends () feature on SMP with barrier () function on up
SMP_WMB ()	WMB () feature on SMP with barrier () function on up
SMP_MB ()	MB () feature on SMP with barrier () function on up
Barrier ()	Prevents the compiler from crossing barriers to optimize loading or storage operations

To make the above small example work correctly, modify the function of thread A with the functions in the table above:

/* * Assume that the operations of A and B in thread a */void thread_a () {    a = 5;    MB ();      /* * MB () ensures that all operations of loading and storing values prior to the loading and storing of values (value is 4) of B are     complete (i.e. a = 5; completed)     * As long as the assignment of a is guaranteed before the assignment of B, Then thread B executes as expected.     */    B = 4;}

12. Summary

This section discusses about 11 kernel synchronization methods, except that large kernel locks are no longer recommended for use, and various other locks have their own scenarios.

Understand the various synchronization methods of the applicable scenarios, in order to properly use them, so that our code under the security of the guarantee to achieve optimal performance.

The purpose of synchronization is to ensure the security of the data, in fact, to ensure the security of shared resources among the various threads, the following according to the situation of shared resources to discuss the choice of 10 synchronization methods.

The 10 synchronization methods are marked with a blue box in the diagram.

Tags: linux-kernel

"Linux kernel design and implementation" reading notes (10)-Kernel synchronization method