Linux2.6-kernel Synchronization

I. What is kernel preemption?

1. If a process is being executed in the kernel state, kernel switching is allowed. This is the kernel preemption.

If a process is running in user mode, it doesn't matter whether the kernel is preemptible.

2. kernel preemption is not allowed when any of the following conditions are met

(1) The kernel is executing the interrupt processing program.

(2) the kernel is executing a soft interrupt or tasklet.

(3) kernel preemption is explicitly prohibited

Kernel preemption is allowed only when an exception handling program is executed and kernel preemption is not explicitly prohibited.

Note:

Kernel preemption may cause process switching, or the process moves another CPU from one CPU, which is not allowed in (1) and (2 ).

3. the Linux kernel is preemptible, and processes in both kernel and user States may be preemptible.

4. The preemptible process is not suspended because it is still in the task_running state, but no longer using the CPU

Ii. critical section and nested kernel control path

1. Nesting of the kernel control path is possible only when the kernel is disconnected.

2. The critical section is a piece of code. At any time, only one kernel control can process the critical section.

The nested execution of the kernel control path causes the critical section to be identified and protected through synchronization.

3. there are three kernel control paths: interrupt handler, exception handler, and deletable function, which can form different nested combinations. Different nested combinations must be protected by different Synchronization Methods.

4. to simplify the synchronization of the kernel control path, the system has made the following provisions:

(1) the same type of Interrupt cannot be generated before the interrupt handler ends -----> the interrupt handler does not have to be reentrant.

(2) The Interrupt Processing Program, Soft Interrupt, and tasklet cannot be preemptible or blocked ------> In the case of a single CPU, you can ignore your own synchronization problems.

(3) The interrupt handler cannot be interrupted by deletable functions or system calls.

(4) The Soft Interrupt and tasklet cannot be staggered on a given CPU -----> only the CPU variables accessed by the soft interrupt and tasklet do not need to be synchronized

(5) The same tasklet cannot be executed on several CPUs at the same time -----> only the data structure accessed by tasklet does not need to be synchronized

3. synchronization technology is used in Linux

1. Each CPU variable

(1) Introduction:

Each CPU variable is an array of data structures. Each CPU of the system corresponds to an element of the array.

One CPU cannot access the array elements corresponding to other CPUs, but can read and write its own elements at will.

(2) Advantages: Each CPU variable can provide protection for concurrent access to its own data from different CPUs

(3) limitations:

Access from Asynchronous functions (Interrupt Processing programs and delayable functions) is not protected. In this case, another synchronization primitive is required.

Kernel preemption must be disabled when accessing each CPU variable.

2. Atomic operations

It makes some "Read-Modify-write" operations Atomic. These operations are called atomic operations.

In addition to the "Read-Modify-write" operation, atomic operations can also be performed on an atomic basis based on other more flexible mechanisms to create critical zones.

3. Optimization and memory barrier

(1) After the source code is converted to assembly, the Assembly command will be re-ordered. If an error occurs before the commands after the synchronization primitive are scheduled to the synchronization primitive, a barrier is introduced.

(2) All synchronization primitives have the function of optimization and memory barrier.

All atomic operations act as memory barriers.

(3) optimization barrier: ensure that the compilation program will not confuse the Assembly commands placed before the primitive and then the Assembly commands placed after the primitive.

Memory barrier: Make sure that the barrier operation is completed before the barrier operation starts.

4. spin lock

If the kernel control path finds that the spin lock is "Open", it gets the spin lock and continues execution.

If the spin lock is "locked", it will "Rotate" around it and execute a compact loop command repeatedly (busy, etc.) until the lock is released.

When a single CPU is used, the spin lock only disables or enables kernel preemption.

5. read/write spin locks

(1) Role: Improve concurrent reading of data structures

(2) principle:

Only when there is no kernel control path to modify the data structure, the read/write spin lock allows multiple kernel control paths to read a data structure at the same time.

If a kernel control path wants to write the data structure, it must first obtain the read/write spin lock.

6. Sequential lock

(1) Role: similar to the read/write spin lock, giving the write with a higher optimization level

(2) Principle: write operations are allowed even if the reader is reading

(3) advantage: the writer will never wait

(4) Disadvantages: readers may read the same data structure multiple times to obtain valid copies.

7. Read-copy-Update (RCU)

(1) Role: allow concurrent execution by multiple readers and writers

(2) features:

Do not use locks

Only protects data structures dynamically allocated and referenced by pointers

In the critical section protected by RCU, no kernel control path can be sleep.

(3) Principle

Reader: indirectly references the memory unit corresponding to the data structure pointer and starts to read the data structure. Before the read operation is completed, the system cannot sleep.

Writer: indirectly references the pointer, generates a copy of the entire data structure, and then modifies the copy. After the modification, the writer changes the pointer so that it points to the modified pointer. At this time, the old copy cannot be released until all readers of the old copy have read it.

(4) Scope of application: RCU is mainly used for network layer and virtual file systems.

8. semaphores

Principle: encapsulate requested and released resources as atomic operations. When the kernel control path attempts to obtain busy resources protected by semaphores, the process is suspended until the resources are released, will be awakened.

Feature: only functions that can sleep can obtain the kernel semaphore. The interrupt handler function and the deletable function cannot use the kernel semaphore.

9. read/write semaphores

(1) features: it is similar to the read/write spin lock, but the waiting process is suspended and not busy.

(2) principle:

Multiple readers can concurrently obtain the read/write semaphores.

The writer must mutually access the protected resources.

A write semaphore can be obtained only when no reader or writer holds a semaphore.

10. Supplemental primitives

Used to solve a special competition between multiple processors

11. Disable local interruption

Protects the data structure accessed by interrupt handlers

In the case of multiple CPUs, it must be combined with the spin-lock phase.

12. Disable and activate delayable Functions

4. How to Select synchronization primitives for different kernel control path nesting methods?

1. Principles for selecting synchronization primitives

(1) The selection of synchronization primitives depends on the types of kernel control paths that access the data structure.

(2) In order to maximize the I/O throughput, minimize the interruption prevention time.

(3) Use as few spin locks as possible to maximize CPU Efficiency

(4) As long as the kernel control path obtains the spin lock (or the read/write lock, sequence lock, and RCU read lock), local interruption or local soft interruption will be disabled, and kernel preemption will be automatically disabled.

2. Select the spin lock Based on the nested mode of the kernel control path.

Kernel control path for Data Structure access	Single-processor Protection	Multi-processor further protection
Exception	Semaphores	None (1)
Interrupted	Local interruption prohibited (2)	Spin lock (3)
Delayable Functions	None (4)	No or spin lock (5)
Exception + interrupt	Local interruption prohibited (6)	Spin lock
Exception + delayable Function	Local Soft Interrupt prohibited (7)	Spin lock
Interrupt + delayable Function	Local interruption prohibited (8)	Spin lock
Exception + interrupt + delayable Function	Local interruption prohibited	Spin lock

Analysis:

(1) semaphores work in exactly the same way on single processor and multi-processor

(2) For a Data Structure accessed by only one interrupt handler, no synchronization primitive is required because each interrupt handler function is executed in a serial manner.

For data structures that will be accessed by multiple interrupt handlers, only local interruptions can be prohibited, because semaphores cannot be used to interrupt the handler, and spin locks may cause the system to be frozen.

(3) In the case of multiple CPUs, the spin lock will not freeze the system

(4) The delayable function is executed serially on a CPU without the need for a synchronization primitive.

(5) Different Processing Methods of delayable functions are different:

Soft Interrupt: spin lock

One tasklet: none (cannot be executed concurrently)

Multiple tasklet: spin lock

(6) The interruption will not be interrupted by exceptions. You only need to ensure that the execution will not be interrupted when exceptions occur.

(7) delayed processing of function execution without exceptions.

Or the latency function is activated by the appearance of the interrupt. There is no need to prohibit interruption. As long as soft interruption is disabled, it can ensure that the execution will not be interrupted by the deletable function when an exception occurs.

(8) delayed function execution may interrupt