Linux multi-thread practice (5) Posix semaphores and mutex locks solve producer and consumer problems

Linux multi-thread practice (5) Posix semaphores and mutex locks solve producer and consumer problems

The difference between Posix and System V semaphores:

System v semaphores can only be used for synchronization between processes, while posix semaphores can be used for synchronization between processes. System v semaphore each PV operation can be N, but Posix semaphore each PV can only be 1. In addition, posix semaphores are named and anonymous (man 7 sem_overview ):

Posix semaphores
Famous semaphores	Unknown semaphores
Sem_open	Sem_init
Sem_close	Sem_destroy
Sem_unlink
Sem_wait
Sem_post

Famous semaphores

 

 

#include 
 
             /* For O_* constants */  #include 
  
           /* For mode constants */  #include 
   
      sem_t *sem_open(const char *name, int oflag);  sem_t *sem_open(const char *name, int oflag, mode_t mode, unsigned int value);  int sem_close(sem_t *sem);  int sem_unlink(const char *name);  
   
  
 

 

Similar to Posix-class IPC usage: The name is identified in the form of/somename and can only have one/, and the total length cannot exceed the NAME_MAX-4 (I. e., 251 ).

Posix famous semaphores need to be created or opened using the sem_open function. PV operations are sem_wait and sem_post, respectively. You can use sem_close to close and delete sem_unlink.

The famous semaphores are used for inter-process synchronization (which can be accessed by name) that does not require shared memory, similar to the System V semaphores.

Anonymous semaphore

 

#include 
 
    int sem_init(sem_t *sem, int pshared, unsigned int value);  int sem_destroy(sem_t *sem); 
 

 

Anonymous semaphores only exist in the memory and require that processes that use semaphores must be able to access the memory. This means they can only apply to threads in the same process, or threads in their address spaces mapped with the same memory content in different processes.

It is stored in a shared memory. For thread sharing, this region can be a global variable. For process sharing, it can be system v shared memory (created by shmget and shmat ing ), it can also be posix shared memory (shm_open creation, mmap ing ).

The anonymous semaphores must be initialized using sem_init. One of the parameters of sem_init determines whether thread sharing (pshared = 0) or process sharing (pshared! = 0). You can also use sem_post and sem_wait to perform operations. Before the shared memory is released, the anonymous semaphore must be destroyed with sem_destroy.

Posix semaphore PV operation

Int sem_wait (sem_t * sem); // P operation int sem_post (sem_t * sem); // V operation

The wait operation reduces the semaphores by 1. If the semaphores count is 0, blocking occurs;

The post operation adds the semaphores to 1. When sem_post is called, if the process is blocked when sem_wait is called, then the process will be awakened and the semaphores increase by 1 in sem_post will be reduced by 1 again by sem_wait;

Posix mutex lock

 

# Include
 
  
Int pthread_mutex_init (pthread_mutex_t * mutex, const pthread_mutexattr_t * mutexattr); // mutex lock initialization. Note: After the function is successfully executed, the mutex lock is initialized to unlocked. Int pthread_mutex_lock (pthread_mutex_t * mutex); // The mutex lock int pthread_mutex_trylock (pthread_mutex_t * mutex); // The mutex lock determines the lock int (pthread_mutex_t * mutex ); // unlock the mutex int pthread_mutex_destroy (pthread_mutex_t * mutex); // remove the mutex
 

 

Mutex lock uses a simple locking method to control atomic operations on shared resources. The mutex lock has only two States: Lock/unlock. You can regard the mutex lock as a global variable in a sense. At the same time, only one thread can master a mutex lock. A thread with a lock status can operate on shared resources. If other threads want to lock a locked mutex lock, the thread will be blocked until the lock thread releases the mutex lock. It can be said that this mutex lock ensures that each thread performs atomic operations on the shared resources in sequence.

A mutex can be divided into a quick mutex lock (default mutex lock), a recursive mutex lock, and an error mutex lock. The major difference between these three locks is whether other threads that do not possess the mutex lock need to block the wait when they want to obtain the mutex lock. A quick lock means that the calling thread is blocked until the thread with the mutex lock is unlocked. Recursive mutex locks can be returned successfully and increase the number of times the call thread locks on the mutex. If an error occurs, the mutex lock is a non-blocking version of the quick mutex lock, it returns an error message immediately.

Solving producer and consumer problems

I will not discuss this classic question any more. Just click baidu. We use pseudocode to streamline the process and then use the above API to implement it easily.

 

Summary: A group of producers and consumers share n circular buffers.

In this case, not only do producers and consumers need to be synchronized, but also each producer and consumer must access the buffer mutually.

Define four semaphores:

Empty -- indicates whether the buffer is null and the initial value is n.

Full -- indicates whether the buffer is full and the initial value is 0.

Mutex1: mutex semaphores between producers. The initial value is 1.

Mutex2: mutex semaphores between consumers. The initial value is 1.

Set the buffer number to 1 ~ N-1 defines two pointers, in and out, which are the pointers used by the producer process and consumer process, pointing to the next available buffer.

The producer process while (TRUE) {produces a product; P (empty); P (mutex1); the product is sent to buffer (in); in = (in + 1) mod n; V (mutex1); V (full);} consumer process while (TRUE) {P (full) P (mutex2); remove the product from buffer (out; out = (out + 1) mod n; V (mutex2); V (empty); consume this product ;}

#include 
 
  #include 
  
   #include 
   
    #include 
    
     #include 
     
      #include 
      
       #include 
       
        #include 
        
         #define ERR_EXIT(m) \ do \ { \ perror(m); \ exit(EXIT_FAILURE); \ } while(0)#define CONSUMERS_COUNT 1#define PRODUCERS_COUNT 1#define BUFFSIZE 10int g_buffer[BUFFSIZE];unsigned short in = 0;unsigned short out = 0;unsigned short produce_id = 0;unsigned short consume_id = 0;sem_t g_sem_full;sem_t g_sem_empty;pthread_mutex_t g_mutex;pthread_t g_thread[CONSUMERS_COUNT + PRODUCERS_COUNT];void *consume(void *arg){ int i; int num = (int)arg; while (1) { printf("%d wait buffer not empty\n", num); sem_wait(&g_sem_empty); pthread_mutex_lock(&g_mutex); for (i = 0; i < BUFFSIZE; i++) { printf("%02d ", i); if (g_buffer[i] == -1) printf("%s", "null"); else printf("%d", g_buffer[i]); if (i == out) printf("\t<--consume"); printf("\n"); } consume_id = g_buffer[out]; printf("%d begin consume product %d\n", num, consume_id); g_buffer[out] = -1; out = (out + 1) % BUFFSIZE; printf("%d end consume product %d\n", num, consume_id); pthread_mutex_unlock(&g_mutex); sem_post(&g_sem_full); sleep(1); } return NULL;}void *produce(void *arg){ int num = (int)arg; int i; while (1) { printf("%d wait buffer not full\n", num); sem_wait(&g_sem_full); pthread_mutex_lock(&g_mutex); for (i = 0; i < BUFFSIZE; i++) { printf("%02d ", i); if (g_buffer[i] == -1) printf("%s", "null"); else printf("%d", g_buffer[i]); if (i == in) printf("\t<--produce"); printf("\n"); } printf("%d begin produce product %d\n", num, produce_id); g_buffer[in] = produce_id; in = (in + 1) % BUFFSIZE; printf("%d end produce product %d\n", num, produce_id++); pthread_mutex_unlock(&g_mutex); sem_post(&g_sem_empty); sleep(5); } return NULL;}int main(void){ int i; for (i = 0; i < BUFFSIZE; i++) g_buffer[i] = -1; sem_init(&g_sem_full, 0, BUFFSIZE); sem_init(&g_sem_empty, 0, 0); pthread_mutex_init(&g_mutex, NULL); for (i = 0; i < CONSUMERS_COUNT; i++) pthread_create(&g_thread[i], NULL, consume, (void *)i); for (i = 0; i < PRODUCERS_COUNT; i++) pthread_create(&g_thread[CONSUMERS_COUNT + i], NULL, produce, (void *)i); for (i = 0; i < CONSUMERS_COUNT + PRODUCERS_COUNT; i++) pthread_join(g_thread[i], NULL); sem_destroy(&g_sem_full); sem_destroy(&g_sem_empty); pthread_mutex_destroy(&g_mutex); return 0;}
        
       
      
     
    
   
  
 

Here is a demonstration of Inter-thread synchronization. Currently, the above program producer has one thread each, but the producer's sleep time is five times that of the consumer, so the consumer will often block on sem_wait (& g_sem_empty, because the buffer is often empty, you can change PRODUCTORS_COUNT to 5, that is, five producer threads and one consumer thread, and the producer's sleep time is five times that of the consumer. It can be seen from the dynamic output that, basically, there is a dynamic balance, that is, five producers have produced five items at once, and the consumer consumes one copy per second, just before the producer continues production.

Spin lock and read/write lock

(1) spin lock (Spin lock)

The spin lock is similar to the mutex lock, but the spin lock does not cause the caller to sleep. If the spin lock has been maintained by other execution units, the caller always loops there to see if the lock owner has released the lock. Therefore, the word "Spin" is named. It is used to solve the mutual exclusion of a resource. Because the spin lock does not cause the caller to sleep, the efficiency of the spin lock is much higher than that of the mutex lock. Although it is more efficient than mutex lock, it also has some shortcomings:
1. The spin lock occupies the CPU all the time, and runs until the lock is obtained. So it occupies the CPU. If the lock cannot be obtained within a short period of time, this will undoubtedly reduce the CPU efficiency.
2. A deadlock may occur when a spin lock is used, and a deadlock may occur when a recursive call is made. calling other functions may also lead to deadlocks, such as copy_to_user (), copy_from_user (), and kmalloc ().
Therefore, we need to use the spin lock with caution. The spin lock is only required when the kernel can be preemptible or SMP. In a single CPU and non-preemptible kernel, the spin lock operation is null. The spin lock applies when the lock user has a short lock retention time.
The usage of the spin lock is as follows:
First define: spinlock_t x;
Then initialize: spin_lock_init (spinlock_t * x); // The spin lock must be initialized before it is actually used.
In kernel 2.6.11, combine definition and initialization into a macro: DEFINE_SPINLOCK (x)

Get the spin lock: spin_lock (x); // It is returned only when the lock is obtained. Otherwise, it is always "Spin"
Spin_trylock (x); // returns true if the lock is obtained immediately. Otherwise, false is returned immediately.
Release lock: spin_unlock (x );

(2) read/write lock

1. As long as no thread holds the given read/write lock for writing, any number of threads can hold the read/write lock for reading
2. Only when no thread holds a given read/write lock for reading or writing can the read/write lock be assigned for writing
3. read/write locks are used to read shared locks, while read/write locks are used to write exclusive locks.
Pthread_rwlock_init
Pthread_rwlock_destroy
Int pthread_rwlock_rdlock
Int pthread_rwlock_wrlock
Int pthread_rwlock_unlock

For more information about linux locks, see the IBM documentation:

Http://www.ibm.com/developerworks/cn/linux/l-cn-mthreadps/index.html