Concurrency and competition in Linux

This post describes the differences between kernel APIs (mainly semaphores and spin locks) provided by Linux kernel to address concurrency-caused competing states, focusing on usage. The Code-level reading is intended to be added.
The competing state caused by concurrent execution of the program is a very complicated aspect in the Linux kernel. For Driver developers of devices, it is very important to be familiar with the concurrent mutex processing mechanism provided by the Linux kernel. The so-called race state, in short, is the behavior of kernel data disorder that may occur when multiple kernel threads operate on the same resource. Sharing data is the root cause of concurrency.
Concurrent source-
I divided the concurrency sources into two major aspects: single processor and multi-processor.
1. single processor
For a single processor, concurrency mainly comes from interrupt, preemption of the kernel and various delay queues.
2. multi-processor
The multi-processor situation is more complex. At the same time, the same code can be executed on different processors at the same time. It is more difficult to process such concurrency.
Kernel mutex-
The driver developer's exact understanding of the mutex mechanism provided by the kernel is the key to writing highly secure code. In the Linux kernel, this kind of facility mainly involves semaphores and spin locks.
1. semaphore
The principle of semaphores is mainly based on the atomic test and set operation on a memory unit. Different processors have different assembly languages to provide this mechanism. Because the memory unit is shared for each processor in a multi-processor system, this mechanism is also applicable to the multi-processor.
Compared with the custom lock, the biggest feature of semaphores is to allow the threads that call it to enter the sleep state. This means that the thread trying to obtain a semaphore will lead to loss of the processor's ownership, that is, process switching.
2. spin lock
The emergence of spin locks was initially designed to solve the mutex problem on the multi-processor. The principle is that the thread trying to obtain the lock performs atomic bit testing and setting operations (atomic BIT Test and set). If no other processor enters the critical section, the current thread gets the lock and enters the critical section. If another processor is in the critical section, the current processor enters the waiting state until other processors unlock the lock. The biggest difference with semaphores is that if the thread trying to enter the critical section cannot get the lock, it will continue to execute the atomic bit test operation without entering the sleep state. In this case, nothing can be done on the processor trying to obtain the lock (except for performing this atomic
Bit test operation), because of this, the thread that requires the lock to end the operation in the critical section in the shortest time.
On a single processor, the spin lock is more accurate. A thread trying to obtain the lock actually notifies the system: When I execute the following code, do not switch me out of the processor. Therefore, the spin lock code on a single processor is actually implementing the mandatory operations on the processor. On a single processor, interruption and preemption are the most likely sources of concurrency. Therefore, in fact, the implementation code of the spin lock on a single processor simply comes down to: disconnection and shutdown preemption. Understanding the implementation code of spin lock requires profound processor-related knowledge and assembly language.
Based on the above implementation principle, a core rule for using spin locks is that threads with locks cannot discard the processor. If it gives up the processor, the other code that obtains the processor thread will have to wait for a long time if it wants to obtain the lock. Worse, it will lead to a system deadlock. A more specific description: on a single processor, a lock-obtaining thread must run in an environment where both interruption and preemption are disabled. In this case, it is impossible to discard the processor passively, only the processor is automatically abandoned. For example, if copy_from_user is called in the mutex code, it is possible to sleep the process in the implementation of this function, if the next scheduled thread tries to obtain the lock, the system enters the deadlock status, in this case, the scheduled thread does not take the initiative to discard the processor (because it is running the code trying to get the lock at this time, it is impossible to call functions that may take the initiative to discard the processor ). Example:
If there are two threads A and B in the system, the processor may be lost when thread a executes the following code:
Spinlock_t g_lock = spin_lock_unlocked; // a global spin_lock variable
Code:
Spin_lock_init (& g_lock );
Spin_lock (& g_lock );
Copy_from_user (...); // Thread a loses the ownership of the processor in this function.
The code of the line B that is scheduled to be executed:
...
Spin_lock (& g_lock); // because g_lock is not released in sleep thread a, thread B will always spin here. Note that both the interrupt and preemption are disabled at this time, that is, it is impossible to discard the processor passively.
The complexity of mutex is exactly here: sleep occurs in many unexpected places. Therefore, when writing code that needs to be executed under the spin lock, you must pay attention to every called function.