General mistakes in parallel computing

This is an old article written in 2013, put on Gegahost.net http://raison.gegahost.net/?p=97

March, 2013General mistakes in parallel computingfiled under:concurrency,software-tags:atomic, cocurrency, data RAC E, dead Lock, Parellel-raison @ 2:51 am

(Original work by Peixu Zhu)

In parallel computing environment, some general mistakes is frequent and difficult to shoot, caused by random CPU Sequenc E in different thread contexts. Most of them is atomic violation, order violation, and dead lock. Studies show that some famous software also has such mistakes, like MySQL, Apache, Mozilla, and OpenOffice.

1. Atomic violation

In Sequent programming, we seldom care the atomic operation, however, in parallel programming, we must remember Atomic ope Rations at first. For example:

[Thread 1]


if (_ptr)         // A
*_ptr = 0; // B


[Thread 2]

_ptr = NULL;        // C

For above code, there's one statement to being executed in thread 1 and thread 2 respectively, it seems the it should be run Ning the statement in thread 1 or thread 2, they should isn't be interlaced. But, in fact, statement in thread 1 are not atomic, at least, it can be divided to step A and B, thus, if it is Arranged to execute in order of A-B-C, it's okay, however, it is also possible being scheduled to run as A-c-b, this would BR ing an unexpected memory access error.

We assume that the statement region in thread 1 is atomic, but it's not true. This is the root of the atomic violation. In many cases, the problem was caused by code modification, for above example, the statement in thread 1 Signment statement at first:
_ptr = &_val;
And later, the code is modified, and the implicit atomicity is broken.

For systems with multiple cores, the problem would be is more complicated, since each core may cache a block of memory respect Ively. For example, core 1 runs thread 1, and Core 2 runs thread 2:
[Thread 1]
_ptr = &_val;

[Thread 2]
_ptr = NULL;

Is they atomic? No, they is not in fact. The ' _ptr ' May was optimized to being register value in one core locally, or it was cached in different core. Thus, the We can not determine the value of ' _ptr '.

To avoid atomic violation, we must make the code region atomic, by locking or atomic operations. Explicit atomic operations on a shared variable was a good habit, since we are noticed by the statement that it's atomicit Y demanded when we try to modify the code.

2. Order violation

Considering below example:
[Thread 1]
_ptr = allocate_memory(); // A

[Thread 2]
_ptr[1] = "right"; // B

If the code is not synchronized, execution order of a-B or b-a is all possible. In such cases, we must synchronize the code block to ensure the order of execution.

3. Dead Lock

Locking is elemental in concurrent programming. If there ' s more than one threads working with more than with one GKFX resource, such as memory block, it is possible tha T each thread owning a resource are waiting for each others resource.
[Thread 1]

lock_a.lock();
a = 0; // A
lock_b.lock();
b = 0; // B
lock_b.unlock();
lock_a.unlock();


[Thread 2]

lock_b.lock();
b = 1; // C
lock_a.lock();
a = 1; // D
lock_a.unlock();
lock_b.unlock();


If the code is running as a-b-c-d, there's no problem, however, if it is running as a-c-b-d, there ' s dead lock. Dead locking requires four conditions:
A. Mutex exclusion
B. Hold and wait
C. No preemption
D. Circular waiting

Breaking at least one of above four condition would break the dead locking.

General mistakes in parallel computing