Efficient development experience of multithreaded programming for Linux

http://www.ibm.com/developerworks/cn/linux/l-cn-mthreadps/

Background

Multithreaded APIs on Linux platforms have subtle and cryptic differences in the multi-threaded API that corresponds to other platforms, such as Windows. Not paying attention to some of these development traps on Linux often leads to poor program problems and deadlocks. In this paper, we summarize the problems of multi-threaded programming in Linux from 5 aspects, and draw up the relevant improvement experience to avoid these pitfalls. We hope that these experiences will help readers get acquainted with the multi-threaded programming of the Linux platform better and faster.

We assume that the reader is already familiar with the basic threading programming Pthread Library API on the Linux platform. Other third-party libraries for threading, such as boost, will not be mentioned in this article. Topics covered in this article include thread management, mutex variables, condition variables, and so on in thread development. The process concept will not be covered in this article.

Back to top of page

Overview of Linux on-line development API

Multithreaded development already has a mature Pthread library support on the Linux platform. The most basic concepts involved in multithreaded development include three points: threads, mutexes, conditions. Among them, the thread operation and the creation of threads, exit, waiting for 3 kinds. The mutex consists of 4 operations, namely, create, destroy, lock and unlock. There are 5 actions for conditional operations: Create, Destroy, Trigger, broadcast, and wait. Other thread extension concepts, such as semaphores, can be encapsulated by the basic operation of the three basic elements above.

threads, mutexes, conditions on the Linux platform the corresponding API can be summarized in table 1. To familiarize readers familiar with Windows threading programming with the API for Linux multithreaded development, we also list the API names in the Windows SDK Library in the table.

Table 1. List of thread functions

Object	Operation	Linux Pthread API	Windows SDK Library corresponding API
Thread	Create	Pthread_create	CreateThread
	Exit	Pthread_exit	ThreadExit
	Wait	Pthread_join	WaitForSingleObject
Mutual exclusion Lock	Create	Pthread_mutex_init	CreateMutex
	Destroyed	Pthread_mutex_destroy	CloseHandle
	Locking	Pthread_mutex_lock	WaitForSingleObject
	Unlock	Pthread_mutex_unlock	ReleaseMutex
Conditions	Create	Pthread_cond_init	CreateEvent
	Destroyed	Pthread_cond_destroy	CloseHandle
	Trigger	Pthread_cond_signal	SetEvent
	Broadcasting	Pthread_cond_broadcast	Setevent/resetevent
	Wait	Pthread_cond_wait/pthread_cond_timedwait	Singleobjectandwait

Multithreaded development already has a mature Pthread library support on the Linux platform. The most basic concepts involved in multithreaded development include three points: threads, mutexes, conditions. Among them, the thread operation and the creation of threads, exit, waiting for 3 kinds. The mutex consists of 4 operations, namely, create, destroy, lock and unlock. There are 5 actions for conditional operations: Create, Destroy, Trigger, broadcast, and wait. Other thread extension concepts, such as semaphores, can be encapsulated by the basic operation of the three basic elements above.

Back to top of page

5 experiences in Linux threading try setting the Recursive property to initialize the Linux mutex

Mutexes are the basic concepts in multithreaded programming and are widely used in development. The sequence of calls is clear and simple: building locks, locking, unlocking, destroying locks. However, it is important to note that, unlike mutual exclusion variables such as the Windows platform, by default, the same thread under Linux cannot recursively accelerate the same mutex, or a deadlock will occur.

The so-called recursive lock, is the same thread in the attempt to the mutex two or more than two times behavior. The code for its scenario on the Linux platform can be seen in Listing 1.

Listing 1. Linux Duplicate Mutex Lock instance

Lock    pthread_mutex_t *themutex = new pthread_mutex_t by default condition;     pthread_mutexattr_t attr;     Pthread_mutexattr_init (&attr);     Pthread_mutex_init (themutex,&attr);     Pthread_mutexattr_destroy (&attr);     Recursive plus lock    pthread_mutex_lock (Themutex);     Pthread_mutex_lock (Themutex);     Pthread_mutex_unlock (Themutex);     Pthread_mutex_unlock (Themutex);

In the above code scenario, the problem will occur in the second lock operation. Because Linux does not allow the same thread recursion to be locked by default, a deadlock occurs during the second lock operation.

Linux mutexes This strange behavior may be useful for certain scenarios, but for most cases it looks more like a bug in the program. After all, recursive locking of the same mutex in the same thread is often required in particular in two development times.

This issue is related to the default recursive property in the mutex. The workaround is to explicitly set the recursive property when the mutex is initialized. Based on this, the above code is actually slightly modified to run very well, just need to set a property when the lock is initialized. Take a look at Listing 2.

Listing 2. Set mutex Recursive property instance

Pthread_mutexattr_init (&attr);     Set Recursive property    Pthread_mutexattr_settype (&ATTR,PTHREAD_MUTEX_RECURSIVE_NP);     Pthread_mutex_init (THEMUTEX,&ATTR);

Therefore, it is recommended to set the recursive property as much as possible to initialize the mutex of Linux, which can solve the problem of the same thread recursion and lock, and avoid the deadlock in many cases. One additional benefit of doing this is to make the lock performance unified under Windows and Linux.

Notice the problem of automatic reset of trigger condition variables on Linux platform

There are two common models for conditional variable placement and reset: The first model is when the conditional variable is set (signaled), and if no thread is currently waiting, its state remains in place (signaled) until a waiting thread enters the trigger. Its status is changed to reset (unsignaled), which is represented by the Auto-set Event on the Windows platform. The status changes are shown in 1:

Figure 1. Condition variable state change process for Windows

The second model is the model used by the Pthread of the Linux platform, and when the condition variable is set (signaled), its state reverts to the reset (unsignaled) state even though no threads are currently waiting. The status changes are shown in 2:

Figure 2. Condition variable state change process for Linux

Specifically, the conditional variable state change model under Pthread on a Linux platform works by calling Pthread_cond_signal () to release a thread that is blocked, regardless of whether the blocked thread exists, the condition is reset. The next thread that is blocked by the condition will not be affected. For Windows, when calling SetEvent to trigger the Event condition of auto-reset, if there is no thread that is blocked by the condition, then the condition will remain in the trigger state until a new thread is blocked by the condition and released.

This discrepancy can be an unexpected embarrassment for programmers who are familiar with the conditional-variable state model on the Windows platform and are developing multithreading on the Linux platform. Imagine the process of achieving a passenger taxi: passengers on the roadside waiting for taxis to call conditions. The taxi came and will trigger the condition, the passengers stop waiting and get on the bus. A taxi can only carry a wave of passengers, so we use a single trigger condition variable. This implementation logic in the first model even if the taxi first arrived, there will be no problem, its procedure 3 shows:

Figure 3. Taxi instance process using the Windows conditional variable model

However, if you follow this approach to Linux, the code might look like Listing 3.

Listing 3. Example of a Linux taxi case code

...//prompt taxi arrival condition variable pthread_cond_t taxicond;  Synchronous lock pthread_mutex_t Taximutex; Passenger arrives waiting taxi void * traveler_arrive (void * name) {cout<< "Traveler:" << (char *) name<< "needs a T Axi now!     "<<endl;     Pthread_mutex_lock (&taximutex);     Pthread_cond_wait (&taxicond, &taxtmutex);     Pthread_mutex_unlock (&taxtmutex);     cout<< "Traveler:" << (char *) name << "Now got a taxi!" <<endl;  Pthread_exit ((void *) 0); }//Taxi arrives void * taxi_arrive (void *name) {cout<< "Taxi" << (char *) name<< "arrives." <<e     Ndl     Pthread_cond_signal (&taxtcond);  Pthread_exit ((void *) 0);     } void Main () {//Initialize taxtcond= Pthread_cond_initializer;     Taxtmutex= Pthread_mutex_initializer;     pthread_t thread;     pthread_attr_t threadattr;     Pthread_attr_init (&AMP;THREADATTR);     Pthread_create (&thread, & Threadattr, taxt_arrive, (void *) ("Jack"));     Sleep (1); PThread_create (&thread, &threadattr, traveler_arrive, (void *) ("Susan"));     Sleep (1);     Pthread_create (&thread, &threadattr, taxi_arrive, (void *) ("Mike"));     Sleep (1);  return 0; }

OK, run it and see the results as shown in Listing 4.

Listing 4. Program result output

Taxi Jack arrives.     Traveler Susan needs a taxi now!     Taxi Mike arrives.     Traveler Susan now got a taxi.

It is shown in procedure 4:

Figure 4. Taxi instance process using the Linux conditional variable model

By comparing the results, you will find that the same logic, the results of running on the Linux platform are completely different. For model one on the Windows platform, Jack drives a taxi to the platform, triggering the condition variable. If there is no customer, the condition variable will maintain the trigger state, which means Jack stops the car and waits. Until Miss Susan came to the platform and waited to find a taxi. Susan took Jack's cab and left, while the condition variable was automatically reset.

But to the Linux platform, the problem came, Jack to the platform to see no one, triggered by the condition variable is directly reset, so Jack queued in the waiting queue. A second later, Miss Susan arrived at the platform and couldn't see Jack waiting there, only to wait until Mike drove by and re-triggered the condition variable, and Susan was on Mike's car. This is not fair to Jack in front of the queuing system, and the crux of the problem is a Bug caused by the automatic reset of conditional variables triggered on the Linux platform.

This model of conditional variables on Linux platforms is hard to say. But in practical development, we can avoid this difference by slightly improving the code. Since this difference occurs only when the trigger is not waiting on the condition variable by the thread, we only need to grasp the timing of the trigger. The simplest way to do this is to add a counter to record the number of waiting threads and check the variables before deciding on the trigger condition variable. The improved Linux functions are shown in Listing 5.

Listing 5. Example of a Linux taxi case code

...//prompt taxi arrival condition variable pthread_cond_t taxicond;  Synchronous lock pthread_mutex_t Taximutex;  Number of passengers, initially 0 int travelercount=0; Passenger arrives waiting taxi void * traveler_arrive (void * name) {cout<< "Traveler:" << (char *) name<< "needs a T Axi now!     "<<endl;     Pthread_mutex_lock (&taximutex);     Increase the number of passengers travelercount++;     Pthread_cond_wait (&taxicond, &taximutex);     Pthread_mutex_unlock (&taximutex);     cout<< "Traveler:" << (char *) name << "Now got a taxi!" <<endl;  Pthread_exit ((void *) 0); }//Taxi arrives void * taxi_arrive (void *name) {cout<< "Taxi" << (char *) name<< "arrives." <<  Endl         while (true) {Pthread_mutex_lock (&taximutex);             The conditional variable if (travelercount>0) {pthread_cond_signal (&taxtcond) is triggered when a traveler is found to be waiting.             Pthread_mutex_unlock (&taximutex);         Break } pthread_mutex_unlock (&taximutex);     } pthread_exit ((void *) 0); }

Therefore, we recommend that you check for a waiting thread before starting the condition variable on the Linux platform, and only trigger the condition variable when it waits.

Notice the unlocking problem of the mutex when the condition returns

When Linux calls pthread_cond_wait to perform conditional variable wait operations, it is necessary to add a mutex variable parameter, which is to avoid competition and starvation between threads. However, when conditions await return, it is important to note that you must not omit to unlock the mutex variable.

When the pthread_cond_wait (pthread_cond_t *cond, pthread_mutex_t *mutex) function on the Linux platform returns, the mutex mutex will be in a locked state. Therefore, if you need to re-access the critical section data later, there is no need to re-lock the mutex on the line. However, the problem with this is that each condition waits for a manual unlock operation to be added in the future. As in the previous article, the Linux code for the passenger waiting for a taxi is shown in Listing 6:

Listing 6. Unlock instance after condition variable is returned

void * Traveler_arrive (void * name) {     cout<< "traveler:" << (char *) name<< "needs a taxi now!" &L t;<endl;     Pthread_mutex_lock (&taximutex);     Pthread_cond_wait (&taxicond, &taxtmutex);     Pthread_mutex_unlock (&taxtmutex);     cout<< "Traveler:" << (char *) name << "Now got a taxi!" <<endl;     Pthread_exit ((void *) 0);  }

This is especially important for developers who are familiar with the multithreaded development of Windows platforms. The signalobjectandwait () function on Windows is a pair of equivalent functions commonly associated with the pthread_cond_wait () function on a Linux platform as a cross-platform programming. However, it is important to note that the state of the two function exits is not the same. On the Windows platform, SignalObjectAndWait (HANDLE A, HANDLE B, ...) method at the end of the call the state of the return is a and B are both set (signaled) state, in a common use method, A is often a Mutex change In this case, when returned, mutex A is in the unlocked state (signaled) and Event B is in the set state (signaled), so for Mutex A, we do not need to consider unlocking the problem. Also, after signalobjectandwait (), if you need to re-access the critical section data, you need to call WaitForSingleObject () to re-lock. This is exactly the opposite of pthread_cond_wait () under Linux.

Linux is important for this extra-unlocked operation of Windows, so be sure to keep it in mind. Otherwise, the conditional wait operation from Windows porting to Linux will definitely deadlock if it forgets the unlock operation after the end.

The absolute time problem of waiting

Timeouts are a common concept in multithreaded programming. For example, when you use Pthread_cond_timedwait () on a Linux platform, you need to specify the timeout parameter so that the caller of the API is only blocked at the specified time interval. But if you're using this API for the first time, the first thing you need to know is the specifics of the timeout parameter in this API (as the title of this section suggests). Let's first look at the definition of this API. See listing 7 for the pthread_cond_timedwait () definition.

Listing 7. pthread_cond_timedwait () function definition

int pthread_cond_timedwait (pthread_cond_t *restrict cond,               pthread_mutex_t *restrict Mutex,               const struct Timespec *restrict abstime);

The parameter abstime is used here to denote a parameter related to the timeout time, but it is important to note that it represents an absolute time, not a time interval value, which triggers a timeout event only if the current time of the system reaches or exceeds the time represented by Abstime. This can be especially confusing for people with threading experience with Windows platforms. Because all API wait parameters (such as signalobjectandwait, etc.) are relative times under the Windows platform,

Suppose we specify a relative time-out parameter such as dwmilliseconds (in milliseconds) to invoke and time-out related functions, so that you need to convert dwmilliseconds to absolute time parameters under Linux abstime use. The usual conversion methods are shown in Listing 8:

Listing 8. Relative time to absolute time conversion instance

/* Get the current time */     struct timeval now;     Gettimeofday (&now, NULL);     /* Add the offset to get timeout value *     /abstime->tv_nsec = now.tv_usec * + (dwmilliseconds% 1000) * 100000 0;     Abstime->tv_sec = now.tv_sec + dwmilliseconds/1000;

The absolute time of Linux is seemingly straightforward, but it is a very obscure pitfall in development. And once you forget the time conversion, you can imagine how frustrating it would be to wait for your mistake: if you forget to convert the relative time to absolute time, it's equivalent to telling the system that the timeout you're waiting for is a period of time past January 1, 1970, so the operating system does not hesitate to give you a The return value of timeout, and then you'll hold your fist and complain about why another synchronization thread is taking so long and plunging into the abyss of searching for time-consuming reasons.

Correctly handle thread-end issues under Linux platforms

Under the Linux platform, one of the issues to be aware of when processing threads end is how to make a thread finish and let its resources be released correctly. On the Linux platform by default, the termination of one thread does not notify or affect other threads, although the threads are independent of each other. But the resources of the terminated thread will not be freed as the thread terminates, we need to call Pthread_join () to get the terminating state of the other thread and release the resources that the thread occupies. The definition of the Pthread_join () function is shown in Listing 9.

Listing 9. Pthread_join function definition

int Pthread_join (pthread_t th, void **thread_return);

The thread that called the function hangs, waiting for the end of the thread represented by th. Thread_return is a pointer to the thread th return value. It is important to note that the thread represented by th must be joinable, that is, in a non-detached (free) state, and only one thread can have a unique call to th pthread_join (). If th is in the detached state, then the Pthread_join () call to TH will return an error.

If you don't care about the end state of a thread at all, you can also set a thread to the detached state, allowing the operating system to reclaim the resources it occupies at the end of the thread. Setting a thread to the detached state can be done in two ways. One is to call the Pthread_detach () function to set thread th to detached state. Its declaration is shown in Listing 10.

Listing 10. Pthread_detach function definition

int Pthread_detach (pthread_t th);

Another approach is to set it to the detached state when the thread is created, initialize a thread property variable first, then set it to the detached state, and finally pass it as a parameter to the thread creation function pthread_create () so that the created thread is directly in the Detached status. The method is shown in Listing 11.

Listing 11. To create a detach thread code instance

.............................     pthread_t       tid;     pthread_attr_t  attr;     Pthread_attr_init (&attr);     Pthread_attr_setdetachstate (&attr, pthread_create_detached);     Pthread_create (&tid, &attr, Thread_function, ARG);

In summary, in order to avoid a potential memory leak problem when the thread's resources are not properly freed when using Pthread, to ensure that the thread is in the detached state at the end of the thread, no need to call Pthread_join () function to recycle the resource.

Back to top of page

Summary and supplement

This article describes in detail the 5 efficient development experiences of multithreaded programming for Linux. Alternatively, you may want to consider some other open source class libraries for threading development.

1. Boost Library

The boost library comes from a member of the C + + standards Committee Class library team who is dedicated to developing a new class library for C + +. Although the library itself is not for multithreading, but it has been developed so far, it has provided a more comprehensive multi-threaded programming API support. The Boost library is more like the Linux Pthread Library for multithreaded support, except that threading, mutexes, conditions, and other thread-development concepts are packaged into C + + classes to facilitate development calls. The Boost library is now well-supported for cross-platform support, not only for Windows and Linux, but also for a variety of commercially available Unix versions. The Boost library will be the first choice if developers want to reduce the difficulty of cross-platform development with a high-stability unified threading programming interface.

2. ACE

ACE full name is ADAPTIVE communication environment, a free, open-source, object-oriented tool framework for developing concurrent access software. Since ACE was originally developed for programming on the Web server side, it also provides comprehensive support for the Threading tool Library. Its supported platforms are also comprehensive, including windows,linux and various versions of Unix. The only problem with ACE is that if it's just for threading, it seems a bit too heavyweight. And its more complex configuration makes it easier for beginners to deploy.

Efficient development experience of multithreaded programming for Linux