Deep analysis of Linux spin locks

original 2016-08-12 tekkamanninja cu technology community Author | Tekkamanninja This article copyright by Tekkamanninja All, if need to reprint, please contact the public number to obtain authorization!
In the process of reviewing hibernation, I want to verify that the spin lock is not dormant, so I write a module that sleeps in the spin lock. But when tested in my ARMV7 single-core CPU (TI's A8 chip), it is not locked, and the spin lock can be obtained multiple times.      The experimental phenomenon and my understanding of spin lock and sleep have a way out. I later put this module on my own PC to test, successfully locked up, indicating that my module does not have a problem in principle. But why is this on arm??? Later, I gave the module to my two colleagues to test, in the Omap3530 will not lock the dead, but in the s3c6410 successfully locked dead. What's going on?? I think it should be the kernel configuration problem, let colleagues will his 6410 kernel configuration to me contrast, found that the configuration is different: 6410 on the spinlock is more than the Config_debug_spinlock spin lock debugging function. So I will own the core of the board also added this configuration, and let colleague Omap3530 kernel also added this configuration to test, the result is normal: Lock dead!! How does a debug option affect the basic function of a spin lock? That means I don't understand the spin lock correctly.     At such times RTFSC is the best solution. I discovered by reading the kernel spin lock source code: if the kernel is configured as an SMP system, the spin lock implements a true spin wait on the SMP system, but for the up system, the spin lock only does the preemption and interrupt operation, without achieving a true "spin". If Config_debug_spinlock is configured, the spin lock is compiled according to the SMP system. But why don't you need a real "spin" spin lock in the UP system? In fact, in understanding the concept and origin of spin lock, this problem is solved. so i re-searched the data about spin lock, and studied the realization of spin lock and related content.
The origin of spin lock spinlock
As we all know, the spin lock was originally designed for SMP systems, enabling the protection of critical areas in multi-processor cases. So in the SMP system, the implementation of the spin lock is intact. however, for up systems, spin locks can be said to be an SMP version of the castrated edition. Because only the spin lock in an SMP system requires a true "spin".
purpose of Spin lock
The spin lock is implemented to protect a short critical section operation code, ensuring that the operation of the critical section is atomic, thus avoiding concurrent competitive adventures.      In the Linux kernel, spin locks are typically used for operations that contain kernel data structures, and you can see that there are spinlock embedded in many kernel data structures, most of which are used to ensure that the atom itself is manipulated, and that it goes through the process of operating such a structure: lock-Operate-unlock. If the kernel control path finds that the spin lock is "on" (can be obtained), it acquires the lock and continues its own execution. Conversely, if the kernel control path Discovery lock is "locked" by a kernel control path running on another CPU, it "rotates" in place, repeatedly executing a compact cyclic detection instruction until the lock is released. Spin lock is a cyclic detection "busy", that is, wait for the kernel nothing to do (in addition to wasting time), the process is kept running on the CPU, so it must protect the critical area of small, and the operation must be short. However, spin locks are usually very handy because many kernel resources lock only 1 milliseconds of time, so waiting for the spin lock to be released does not consume too much CPU time.
spin lock needed to do the work
The spin lock should block any concurrent interference that occurs during code operation from the point of view of guaranteeing the atomicity of the critical section. These "disturbances" include: 1. Interrupts, including hardware interrupts and software interrupts (required only if the interrupt code may access the critical section) such interference exists in any system, and an interruption of the arrival of the interrupt routine is performed, and if the critical section is accessed in the interrupt routine, the atomicity is broken. So if there is code in some interrupt routine that accesses a critical section, it must be protected with spinlock. Different types of interrupts (hardware interrupts and software interrupts) correspond to different versions of the spin lock implementation, which contains code that interrupts disabling and opening. However, if you guarantee that no interrupt code will access the critical section, then use the spin lock API without the interrupt disable. 2. Kernel preemption(only available in the preempted kernel) in the kernel after 2.6, kernel preemption is supported and configurable. This makes the up system similar to SMP, with concurrency in the kernel state. In this case, it is necessary to avoid the concurrency caused by preemption when entering the critical section, so the solution is to disable preemption (preempt_disable () When the lock is added. ), open preemption (preempt_enable () when unlocking, and note that a preemption schedule is performed at this time. 3. Other processors access to the same critical section(SMP-only system) in SMP systems, multiple physical processors work concurrently, resulting in the possibility that multiple processes are physically concurrent. This requires a flag in memory, and every code that needs to enter the critical section must check this flag to see if a process is already in this critical section.      In this case, the code that checks the flags must also be guaranteed to be atomic and fast, which requires a fine-grained implementation, and normally each architecture has its own assembly implementation, guaranteeing the atomicity of the check. Some people will think that spin-lock spin detection can be implemented with for, the idea "Too Young, Too simple, sometimes naive"! You can theoretically use C to explain, but if you use for, at least there are two questions:
(1) How do you ensure that other processors do not access the same logo at the same time under SMP? (i.e. exclusive access to the flag)
(2) must ensure that each processor does not go to read the cache but the real memory of the flag (can be implemented, programming can be used Volitale)
to solve this problem at all, it is necessary to implement the physical memory address exclusive access at the bottom of the chip, and use special assembly instruction access on the implementation. See the implementation analysis for spin locks in resources. ARM, for example, starts with the ARM architecture instruction set with SMP (V6, V7), and uses the Ldrex and Strex instructions to achieve true spin waiting.
Spin lock Operation composition
Based on the introduction, we can easily know the composition of spin lock:

• Interrupt control (required only if the interrupt code may access the critical section)
• Preemption control (only required in a preempted kernel)
• Spin lock flag Control (SMP system only required)

Interrupt control is based on code access to the critical section and different variants are selected during programming, some of which are not. The preemption control and Spin lock flag control is determined at compile time, depending on the kernel configuration (whether kernel preemption is supported) and the hardware platform (whether it is SMP). If not, the corresponding control code is compiled into an empty function. For non-preemptive cores, each critical section protected by a spin lock has an API that prohibits kernel preemption, but is empty.  Since the up system does not have physical parallelism, it is possible to castrate the spin-off part, leaving the preemption and interrupt operations. Here you can actually explain why the experiment I started with was completely different from what I expected:
Since the up system (in the case of not configuring Config_debug_spinlock), there is no spin-lock control at all, it is possible to get a spin lock multiple times (this programming is wrong, but I want to see the wrong phenomenon).
for one of these doubts:
1, in the version with disabled interrupts, since the interruption has been disabled, on this processor will not be interrupted, disable preemption is redundant?
(1) Disabling interrupts can avoid preemption scheduling due to interrupts, but if preempt_disable () is present in the critical section of the spin lock protection, and preempt_enable (); This will cause a preemption schedule in preempt_enable ().
(2) Avoid other processors in the SMP system to execute the scheduler so that the process of the processor will be dispatched.?????
I am not very sure about this question, and there will be an accurate answer after studying the dispatch system in depth.
rules for the use of spin lock variants
Whether a preemptive up, non-preemptive up or SMP system, as long as a certain class of interrupt code may access the critical section, you need to control the interruption, to ensure the atomicity of the operation. Therefore, this and the module code in the critical section of the access is also related, whether it is possible to operate the critical section in the interrupt, only the programmer will know. So there are spin lock variants for different interrupt types in the spin lock API:
The critical section is not manipulated in any interrupt routines:

static inline void Spin_lock (spinlock_t *lock) static inline void Spin_unlock (spinlock_t *lock)

If you operate a critical section in a software interrupt:

static inline void Spin_lock_bh (spinlock_t *lock) static inline void Spin_unlock_bh (spinlock_t *lock)

BH stands for bottom half, which is the bottom half of the interrupt, because the bottom half of the core interrupt is typically handled by software interrupts (Tasklet, etc.).

If the critical section is operating in a hardware interrupt:

static inline void Spin_lock_irq (spinlock_t *lock) static inline void Spin_unlock_irq (spinlock_t *lock)

If you need to save the interrupt state while controlling the hardware interrupt:

Spin_lock_irqsave (lock, flags) static inline void Spin_unlock_irqrestore (spinlock_t *lock, unsigned long flags)

These cases seem a bit simple, and I found an article on the Internet using the rules (Spinlock) explained in a classic, thorough, very detailed. I made a slight change, reproduced as follows:
There are several versions of the spin lock and release spin lock, so it is necessary to let the reader know what version of the macro to use to obtain and release the lock in what case. ifShared resources that are protected context access only in process context and soft interrupts (including Tasklet, timer), when a shared resource is accessed in the context of the process, it may be interrupted by a soft interrupt, which may enter the soft interrupt context to access the protected shared resource, so access to the shared resource must use SPIN_LOCK_BH and spin_unlock_bh to protect。 Of course, using SPIN_LOCK_IRQ and SPIN_UNLOCK_IRQ as well as Spin_lock_irqsave and Spin_unlock_irqrestore can also, they fail the local hard interrupt, and the fail hard interrupt implicitly also fails the soft interrupt. But using SPIN_LOCK_BH and SPIN_UNLOCK_BH is the most appropriate, it is faster than the other two.
ifShared resources that are protected access only in two or more tasklet or timer contexts, then access to the shared resource only need to use Spin_lock and spin_unlock to protect, you do not have to use the _BH version, because when the Tasklet or timer runs, it is not possible to have other Tasklet or a timer running on the current CPU. ifShared resources that are protected access only in one Tasklet or timer context, so no spin lock protection is required, because the same tasklet or timer can only run on one CPU, even in an SMP environment. The Tasklet actually binds the Tasklet to the current CPU when the tasklet_schedule tag is called, so the same tasklet is never likely to run on the other CPU at the same time. The timer is also used to Add_timer to the current CPU when it is added to the timer queue, so the same timer will never run on other CPUs. Of course the same tasklet with two instances running on the same CPU at the same time is more unlikely. ifProtected shared resources are accessed only by a soft interrupt (except Tasklet and timer), so this shared resource needs to be protected with spin_lock and Spin_unlock, because the same soft interrupt can run on different CPUs at the same time. ifProtected shared resources are accessed in two or more soft interrupt contexts, so this shared resource will of course need to be protected with Spin_lock and spin_unlock, with different soft interrupts running on different CPUs at the same time. ifShared resources that are protected in soft interrupts (including Tasklet and timer) or process contexts and hard interrupt context access, a hard interrupt may be interrupted during a soft interrupt or process context access, thereby accessing the shared resource in a hard interrupt context, so that the process or soft interrupt context need to use SPIN_LOCK_IRQ and SPIN_UNLOCK_IRQ to protect access to shared resources。 and what version to use in the interrupt handling handle, depending on the situation, if only one interrupt handle handle accesses the shared resource, then in the interrupt handling handle requires only Spin_lock and Spin_unlockTo secure access to shared resources. Because during the execution of the interrupt handling handle, it is not possible to be interrupted by a soft interrupt or process on the same CPU. But if there are different interrupt handling handles to access the shared resource, you need to do so in the interrupt handling handle using SPIN_LOCK_IRQ and Spin_unlock_irqTo protect access to shared resources. in the case of SPIN_LOCK_IRQ and SPIN_UNLOCK_IRQ, it is entirely possible to replace the Spin_lock_irqsave and Spin_unlock_irqrestore, which of the specific should be used depends on the situation, It is better to use SPIN_LOCK_IRQ if you can be sure that interrupts are enabled before access to shared resources. Because it's faster than Spin_lock_irqsave, but if you're not sure if the interrupt is enabled, then using Spin_lock_irqsave and Spin_unlock_irqrestore is better, This is because it will revert to the interrupt flag before accessing the shared resource instead of directly enabling the interrupt. Of course, in some cases You need to interrupt when accessing shared resources, and you must interrupt enable when you are done, such a situation using SPIN_LOCK_IRQ and Spin_unlock_irqBest.
Spin_lock is used to block simultaneous access to shared resources by execution units on different CPUs and to non-synchronous access to shared resources due to mutual preemption of different process contexts, while interrupt invalidation and soft interrupt invalidation are designed to prevent soft interrupts on the same CPU or interrupt non-synchronous access to shared resources.
The above is my understanding of the spin lock and the use of the summary, the implementation of the spin lock, in fact, there have been such articles on the Internet, I do not nonsense. Since spin locks involve kernel preemption, it is best to learn about the following preemption.

Deep analysis of Linux spin locks