How to Ensure the atomicity of interlocked APIs

The previous article mentioned how to use interlocked API to design system-level logs. Interlocked API can provide atomic access to memory variables shared among multiple threads. Some CPUs directly support these operations on the hardware layer, such as X86 architecture CPU after 80386, xchg, xadd, cmpxchg, and other commands to lock the bus during memory access. For example, the implementation of interlockedexchangeadd on x86 is as follows:

1. Long winapi interlockedexchangeadd (plong addend, long value)
2. {
3. _ ASM {
4. MoV ECx, addend
5. MoV eax, Value
6. Xadd [ECx], eax
7. }
8. }

The xadd command is used directly to complete the exchange.
The problem is that CPUs in other architectures do not provide similar commands to lock the bus. How can we ensure the atomicity of interlocked APIs? For the ARM architecture, the implementation of interlockedexchangeadd on Windows CE is as follows:

1. Leaf_entry interlockedexchangeadd
2. LDR R12, [R0]
3. Add R2, R12, r1
4. STR R2, [R0]
5. MoV r0, R12; (R0) = Return Original Value
6. BX lR
7. Entry_end interlockedexchangeadd

Translated into C language:

1. Long interlockedexchangeadd (plong addend, long value)
2. {
3. Long oldval = * target; // LDR R12, [R0]
4. Long newval = oldval + value; // Add R2, R12, r1
5. * Target = newval; // STR R2, [R0]
6. Return oldval; // mov r0, R12; BX lR
7. }

It is a common function. In a multi-threaded environment, such an implementation is insufficient to guarantee atomicity. For example, if you have a global variable g_lvar, thread 1 and thread 2 will change its value:

1. Long g_lvar = 0;
3. Void thread_entry ()
4. {
5. Long oldval = interlockedexchangeadd (& g_lvar, 1 );
6. }

What happens when interlockedexchange cannot guarantee atomicity? Normally, after the two threads run the thread_entry function, the value of g_lvar is 2. Now let's imagine this situation: thread 1 executes the first sentence in interlockedexchangeadd (oldval is 0 at this time), the time slice is used up, and the thread scheduler wakes up thread 2, thread 2 executes thread_entry smoothly, and g_lvar is 1. The thread scheduler switches back to thread 1 for execution. Because the local variable oldval in thread 1 is 0, * target = newval = oldval + value = 0 + 1 = 1, therefore, after thread_entry of thread 1 is complete, the value of g_lvar is still 1 !! The problem is that the execution of interlockedexchangeadd may be interrupted by other threads, and the operands may be changed by other threads. interlockedexchangeadd itself cannot predict this, that is, interlockedexchangeadd's atomicity cannot be guaranteed.
How does Windows CE solve this problem? The answer is the scheduler! Before the CPU is switched to another context for execution (for example, the current task is interrupted or a data abort occurs), The scheduler checks whether the original task runs in the address range of interlocked API, if yes, the scheduler changes the instruction register to the entry address of the interlocked API. In our example, when interlockedexchangeadd runs out of the first time slice, the scheduler finds that the instruction register of this thread is between the interlockedexchangeadd API, so it resets the instruction register back to the entry address of interlockedexchangeadd, switch to thread 2 for execution. When thread 1 is running after thread 2 is complete, the value of oldval is retrieved from * target (that is, g_lvar, which has been changed to 1 by thread 2). In this way, after thread 1 is complete, the value of g_lvar is changed to 2. Note that the scheduler does not guarantee that the execution of interlocked APIS is not interrupted. It ensures that a complete execution of interlocked APIS is not interrupted by other tasks! It should also be noted that this method does not work on a multi-processor or multi-core system. A memory access mechanism that locks the bus must be provided at the hardware level.