"Reprint" of inter-process communication between Linux-semaphores

Source:Inter-process communication for Linux-semaphores

Inter-process communication for Linux-semaphores

版权声明：本文章内容在非商业使用前提下可无需授权任意转载、发布。转载、发布请务必注明作者和其微博、公众号地址，以便读者询问问题和甄误反馈，共同进步。微博ID：orroz公众号：Linux系统技术

Objective

Semaphores are also called semaphores, and some call it a signal set, which is called the "Advanced Programming of UNIX environment" and is still called the semaphore. Its English is semaphores, the intention is "semaphore" "signal" meaning. Because it contains the keyword "signal" in its name, it is easy to confuse another signal with signal. First of all, it is emphasized that the semaphore and signal signals in a Linux system are two completely different concepts. We will explain in detail the signal signal in other articles. This article can help you learn:

1. What is the xsi signal volume?
2. What is PV operation and its application.
3. What is a POSIX semaphore?
4. The operation method of signal volume and its realization.

We already know that file locks are necessary for multi-process shared files, and locking a file can prevent a "race condition" when multiple processes are accessing the file. Semaphores provide a similar ability to handle multiple processes or even multithreading in different states to compete for shared resources. It acts like a semaphore at a crossroads or a ship, and is used to coordinate the access of multiple execution processes to critical sections. But essentially, the semaphore actually implements a set of primitives that can implement a lock-like function, and we can use it not only to implement locks, but also to implement other behaviors, such as classic PV operations.

There are two main types of signals implemented in Linux environments. Depending on the standard, they are similar to shared memory, a set of XSi semaphores, a set of POSIX semaphores. Below we use them to implement a set of similar file lock method, to simply look at their use.

Please scan the QR code, thank you for your donation!

XSi Signal volume

XSI signal is a kernel implementation of a counter, can do a reduction of the counter operation, and operating in accordance with a number of basic operating principles, namely: the counter is added to the immediate return, do reduce the operation to check whether the current value of the counter is reduced enough? (If the reduction is less than 0) if enough, the reduction will not be blocked, if not enough, then the blocking wait until enough to reduce. Here we first give a prototype of its related operation methods:

#include <sys/sem.h>int semget(key_t key, int nsems, int semflg);

You can use Semget to create or open a semaphore array that has already been created. As explained in XSi shared memory, we should already know that the first parameter key is used to identify the semaphore within the system. In addition to using Ftok generation, you can also use Ipc_private to create a semaphore without a key. If the specified key already exists, it means that the semaphore is opened, when the Nsems parameter is specified as the 0,SEMFLG parameter is also specified as 0. The Nsems parameter indicates that the number of semaphores in this array is several when a semaphore array is created. We can implement more complex semaphore functions through arrays of multiple semaphores. The last SEMFLG parameter is used to specify the flag bit, mainly: IPC_CREAT,IPC_EXCL and permissions mode.

#include <sys/types.h>#include <sys/ipc.h>#include <sys/sem.h>int semop(int semid, struct sembuf *sops, size_t nsops);int semtimedop(int semid, struct sembuf *sops, size_t nsops, const struct timespec *timeout);

Use the SEMOP to manipulate the array of semaphores. NSOPS specifies that several elements in an array are manipulated, and only one semaphore in an array is specified as 1. All parameters of the operation are defined in a SEMBUF structure, with the following contents:

unsigned short sem_num;  /* semaphore number */short          sem_op;   /* semaphore operation */short          sem_flg;  /* operation flags */

SEM_FLG parameters that can be specified include ipc_nowait and Sem_undo. When the Sem_undo is established, the process exits automatically and it will UNDO its operation on the semaphore. The operation of the Semaphore will act on the specified number of Sem_num semaphores. The number of the 1th semaphore in a semaphore set starts at 0. Therefore, for a signal set with only one semaphore, this sem_num should be specified as 0. The sem_op is used to specify the operation of the semaphore, there are three kinds of operations:

Positive action: The value of the semaphore counter (semval) is added to the operation.

0 Value operation: There is no effect on the value of the counter and requires that the process must have read access to the semaphore. This behavior is actually a "wait counter of 0" Operation: If the counter has a value of 0, the operation can be returned immediately. If it is not 0 and the SEM_FLG is set to Ipc_nowait, the 0 value operation is not blocked, but returns immediately, and errno is set to Eagain. If it is not 0 and the ipc_nowait is not set, the operation blocks until the counter value becomes 0, at which point the semncnt value of the associated semaphore is incremented by 1, which is used to record how many processes (threads) are waiting on this semaphore. In addition to the blocking stop when the counter becomes 0, there are other conditions that can cause the stop waiting: The semaphore is deleted, the SEMOP operation fails, and the errno is set to EIDRM. The process is interrupted by the signal (signal), errno will be set to EINTR, and the cut semzcnt will be reduced to normal.

Negative action: The counter is reduced, and the process must have write permission on the semaphore. If the value of the current counter is greater than or equal to the absolute value of the specified negative value, SEMOP can return immediately, and the value of the counter is set to the result of the subtraction operation. If the absolute value of the SEM_OP is greater than the value of the counter semval, then there is currently not enough reduction, and the test if SEM_FLG set the IPC_NOWAIT,SEMOP operation will still return immediately and errno is reset to Eagain. If Ipc_nowait is not set, it will block until the following conditions occur:

1. The value of the semval is greater than or equal to the absolute value of Sem_op, which indicates that there is enough subtraction.
2. The semaphore is removed and Semop returns EIDRM.
3. The process (thread) is interrupted by the signal and SEMOP returns EINTR.

These behaviors are basically similar to the 0 value operation. SEMTIMEDOP provides a structure with a timeout mechanism in order to implement a wait timeout. Observing the behavior of SEMOP we will find it necessary to assign a value to its default counter semval after a semaphore has been created. So, we need to use SEMCTL to do the assignment before semop.

int semctl(int semid, int semnum, int cmd, ...);

This call is a variable parameter implementation, and the specific parameters vary according to the CMD. In general use, we mainly learn to use it to change the value of semval and to view and modify the properties of SEM. The associated cmd are: Setval, Ipc_rmid, Ipc_stat.

An example of a simple modification to semval:

semctl(semid, 0, SETVAL, 1)；

This call can set the Semval value of the specified SEM to 1. More specific parameter explanation you can refer to Man 2 semctl.

These are the primitive meanings of semaphore definitions. If it implements a mutex-like operation, then we can initialize a semaphore with a default counter value of 1, minus 1 when someone does a lock operation, plus 1 for unlocking. So for a semaphore counter that has been reduced by 1, another lock will cause the block to wait until the lock is unlocked before someone else locks it.

Let's take a look at their use, we use SEM to implement a set of mutexes, which in addition to locking the file can also be used to lock the shared memory, we can use it to protect the above shared memory when using the critical section. We use the XSI shared memory code case as an example:

此处代码由于篇幅字数限制删除。源代码可以在http://liwei.life上原文中看到。

The correct execution results can be obtained at this time:

[[email protected] sem]$ ./racing_xsi_shm shm_p: 100

You can think for yourself how to use semaphores to improve the operation of all the locks and add the following methods:

1. Implement Trylock.
2. Implement a shared lock.
3. In the case of a shared lock, the implementation looks at how many people are currently sharing the same lock.

The limit for XSI semaphores in the system is placed in a file with the path:/proc/sys/kernel/sem. There are 4 limit values in the file, each of which means:

SEMMSL: The maximum number of semaphores that can be in a semaphore set (semaphore set). This restriction is actually the upper limit of the second parameter of the Semget invocation.

Semmns: The maximum number of semaphores in a system that can be in all semaphores.

SEMOPM: You can use the SEMOP system call to specify the operand limit. This is actually the upper limit of the number of sem_op in the struct of the second argument in the SEMOP call.

Semmni: The ID number limit for semaphores in the system. is the maximum number of semaphore sets.

PV Operation Primitives

The PV operation is one of the key elements in the operating system principle, and according to the description of the mutex function described above, our mutex is actually a typical PV operation. The lock behavior is the p operation, and the unlock is the V operation. PV operation is one of the basic functions that the computer operating system needs to provide. Initially it was used to implement the functional primitives of a multitasking operating system on a computer system with only 1 CPUs. Imagine that multitasking means that multiple processes can be executed simultaneously in the system, but there is only one CPU, which means that at one point there is actually only one process that consumes the CPU, while the rest of the process is waiting. Based on this consideration, in 1962, Dixtra introduced the design of the PV operation Primitive in the system to realize the control primitives of CPU resource in multi-process. After understanding the mutex, we can realize that the critical section code snippet is actually similar to the environment in which the multi-process uses a CPU, both of which are limited resources under competitive conditions. For such resources, it is necessary to use the PV operation primitives for control.

Please scan the QR code, thank you for your donation!

According to this idea, we will expand to see an application. We all know that the current computer is mostly multicore or even multi-CPU scenarios, so many computing tasks can be executed concurrently, it can undoubtedly increase the computational power. Suppose we use a multi-process approach for concurrent operations, how many processes are appropriate? Although this problem will vary depending on the application scenario, if it is assumed to be an extremely CPU-intensive operation, then there is no doubt that a few CPUs should be associated with several processes. At this point, if the number of concurrent too many, it will increase the scheduling cost resulting in the overall swallow metric down, and too little can not take advantage of multiple CPU cores. The PV operation is exactly a programming primitive that can implement a similar approach. Let's assume an application model where the application finds prime numbers from 10010001 to 10020000 numeric ranges. In the case of concurrency, we can consider each of the numbers to be judged by a process to calculate, but this will undoubtedly make the number of processes is much larger than the average number of computer CPUs. So we can control the number of simultaneous operations by using the PV operation primitive when the process is generated. The implementation of the PV primitive is not very different from the above mutex, for the mutex, the initial value of the counter is 1, and for this PV operation, is nothing more than the value of the counter is set to the current computer core number, the specific code is implemented as follows:

[[email protected] sem]$ cat sem_pv_prime.c#include <stdio.h> #include <stdlib.h> #include < errno.h> #include <unistd.h> #include <sys/ipc.h> #include <sys/sem.h> #include <sys/shm.h> #include <sys/types.h> #include <sys/wait.h> #include <signal.h> #define START 10010001#define END 10020000#define nproc 4static int pvid;int mysem_init (int n) {   int semid;    semid = Semget (ipc_priv ATE, 1, ipc_creat|0600);    if (Semid < 0) {       perror ("Semget ()"),        return-1; &nbs P  }    if (Semctl (semid, 0, Setval, N) < 0) {       perror ("Semctl ()");   &NBSP ;    return-1;    }    return Semid;} void Mysem_destroy (int pvid) {   semctl (pvid, 0, ipc_rmid);} int P (int pvid) {   struct sembuf sbuf;    sbuf.sem_num = 0;    sbuf.sem_op =-1;   &NB Sp;sbuf.SEM_FLG = 0;    while (Semop (Pvid, &AMP;SBUF, 1) < 0) {       if (errno = eintr) {    &NB Sp      continue;        }        perror ("Semop (P)");        return-1;    }    return 0;} int V (int pvid) {   struct sembuf sbuf;    sbuf.sem_num = 0;    sbuf.sem_op = 1;   &nbs P;SBUF.SEM_FLG = 0;    if (Semop (Pvid, &AMP;SBUF, 1) < 0) {       perror ("Semop (v)");       & nbsp;return-1;    }    return 0;} int Prime_proc (int n) {   int I, J, flag;    flag = 1;    for (i=2;i<n/2;++i) {  &N Bsp    if (n%i = = 0) {           flag = 0;           &NBSP;B Reak;        }    }    if (flag = = 1) {       printF ("%d is a prime\n", N);    }    /* Sub-process determines the current number before exiting the V operation */  &NBSP;V (Pvid);    exit (0);} void Sig_child (int sig_num) {   while (waitpid ( -1, NULL, Wnohang) > 0);} int main (void) {   pid_t pid    int i    /* use signal processing to recycle when a subprocess exits to prevent many zombie processes */  &NBS P;if (Signal (SIGCHLD, sig_child) = = Sig_err) {       perror ("signal ()");        e XIT (1);    }    pvid = Mysem_init (Nproc);    /* each number that needs to be calculated opens a sub-process for judgment */   for (i=start;i<end;i+=2) {       /* when creating child processes Perform P operations. */       p (pvid);        pid = fork ();       &NBSP;IF (PID < 0) {           /* If creation fails then v action */    &NB Sp     &NBSP;V (pvid);            perror ("fork ()");            exit (1);       &NBSP;}        if (PID = = 0) {           /* Create sub-processes make this number judgment */& nbsp          prime_proc (i);        }    }    /* this waits for all the numbers to be exhausted to prevent the operation from being Mysem_destroy to the last parent process, resulting in the last four sub-processes V-operation times wrong */& nbsp  while (1) {sleep (1);};    mysem_destroy (Pvid);    exit (0);}

The execution logic of the entire process group can be described as a parent process that requires operations to determine all occurrences of a prime number in the range of 10010001 to 10020000 digits, with each count of a number of ways to open a child process. In order to control the number of sub-processes at the same time do not exceed the number of CPUs, so requested a value for the number of CPUs of the semaphore counter, each create a sub-process, on the counter to do p operation, the sub-process operation is rolled out to the counter to do v operation. Because the p operation is blocked when the counter is 0, until another child process exits with a V operation to add 1 to the counter, the entire process group does not produce a task that is larger than the number of CPUs of the child process.

This code uses a signal processing method to reclaim sub-processes in case there are too many zombie processes, and this programming method is more used in daemon. The problem with this approach is that if the parent process does not recycle all the child processes before exiting, the parent process will delete the semaphore before the last few child processes have finished executing, causing the last few sub-processes to perform a V operation with an error. Of course, we can adopt a more elegant way of process processing, but that is not the content of this article to highlight the explanation, we can self-improvement of the relevant methods. The general daemon process normally does not actively exit the parent process, so there is no such problem.

POSIX signal Volume

POSIX provides a new set of Semaphore primitives whose prototypes are defined as follows:

#include <fcntl.h> #include <sys/stat.h>#include <semaphore.h>sem_t *sem_open(const char *name, int oflag);sem_t *sem_open(const char *name, int oflag, mode_t mode, unsigned int value);

Use Sem_open to create or access a POSIX semaphore that has already been created. When created, you can use the value parameter to assign a value directly to it.

int sem_wait(sem_t *sem);int sem_trywait(sem_t *sem);int sem_timedwait(sem_t *sem, const struct timespec *abs_timeout);

The sem_wait will decrement the specified semaphore, and if the semaphore's original value is greater than 0, the minus operation returns immediately. If the current value is 0, the sem_wait will block until it can be reduced.

int sem_post(sem_t *sem);

The sem_post is used to operate the semaphore. This causes a process that has already used sem_wait to return to the semaphore.

int sem_getvalue(sem_t *sem, int *sval);

The sem_getvalue is used to return the value of the current semaphore to the memory address pointed to by Sval. If there is currently a process using sem_wait to wait for this semaphore, POSIX can allow two returns, one to return 0, and the other to return a negative value, the absolute value of which is the number of waiting processes. The default implementation of Linux is to return 0.

int sem_unlink(const char *name);int sem_close(sem_t *sem);

Using Sem_close, you can turn off a semaphore within the process, and Sem_unlink can delete the semaphore in the system.

POSIX semaphore implementations are clearer and more complex, compared to XSI semaphores, but more flexible and more widely used. In the XSI semaphore, the addition and subtraction of the counter is implemented by the Semop method and a sembuff structure, but a clearer definition is given in POSIX: using the Sem_post function can increase the value of the semaphore counter, using the Sem_ Wait can reduce the value of the counter. If the value of the counter is currently 0, the sem_wait operation will block to a value greater than 0.

POSIX semaphores also provide two ways to implement, named Semaphores and anonymous semaphores. This is a bit like the xsi way to use Ftok file path creation and Ipc_private way to create the difference. But the presentation is not quite the same:

Named semaphore:

A named semaphore is actually a semaphore with a file name. Similar to POSIX shared memory, semaphores also create a file in the/dev/shm directory, which is a named semaphore if there is a file name. Other processes can use this semaphore through the Sem_open method through this file name. In addition to accessing a named semaphore, the Sem_open method can also create a semaphore. Once created, you can use Sem_wait, Sem_post, and other methods to do so. It is important to note that a named semaphore is completely deleted after it is closed with Sem_close and the file name is deleted using Sem_unlink.

Anonymous signal Volume:

An anonymous semaphore is just a section of memory and does not have a file name corresponding to it. Anonymous semaphores are initialized with Sem_init and destroyed using Sem_destroy (). The operation method is the same as the named semaphore. The initialization method of anonymous memory is not the same as Sem_open, Sem_init requires that an existing memory be initialized rather than a file generated under/DEV/SHM. This requires that if the semaphore is used in multiple threads in a process, then it should be in the same memory area as the global variables that these threads should be able to access, or the memory that malloc allocates. If it is shared across multiple processes, then this memory should itself be a piece of shared memory (memory requested using MMAP, Shmget, or Shm_open).

The other methods involved in POSIX shared memory should also be relatively simple, with more detailed help referring to the related Man manual, and below we give two code examples using named and anonymous semaphores:

Named semaphores use:

[[email protected] sem]$ cat racing_posix_shm.c#include <unistd.h> #include <stdlib.h> #include < stdio.h> #include <errno.h> #include <fcntl.h> #include <string.h> #include <sys/file.h># Include <wait.h> #include <sys/mman.h> #include <sys/stat.h> #include <semaphore.h> #define  COUNT 100#define Shmpath "/SHM" #define Sempath "/sem" Static sem_t *sem;sem_t *mylock_init (void) {   sem_t * RET;    ret = Sem_open (Sempath, o_creat| O_EXCL, 0600, 1);   &NBSP;IF (ret = = sem_failed) {       perror ("Sem_open ()"),        return N ULL;    }    return ret;} void Mylock_destroy (sem_t *sem) {   sem_close (SEM);    sem_unlink (Sempath);} int Mylock (sem_t *sem) {   while (sem_wait (SEM) < 0) {       if (errno = eintr) {&nbsp ;          continue;        }     &nbsp  perror ("sem_wait ()");        return-1;    }    return 0;} int Myunlock (sem_t *sem) {   if (sem_post (SEM) < 0) {       perror ("Semop ()");   &N Bsp    return-1;    }}int do_child (char * shmpath) {   int interval, SHMFD, ret;    int *shm_p;     SHMFD = Shm_open (Shmpath, O_RDWR, 0600);    if (SHMFD < 0) {       perror ("Shm_open ()"),        exit (1); &nbs P  }    shm_p = (int *) mmap (NULL, sizeof (int), prot_write| Prot_read, map_shared, SHMFD, 0);    if (map_failed = = shm_p) {       perror ("mmap ()"),        exit (1); nbsp  }    /* critical section */   mylock (SEM);    interval = *shm_p;    interval++;    usleep (1);    *shm_p = interval;    myunlock (SEM);   /* critical section */   munmap (shm_p, sizeof (int));    close (SHMFD);    exit (0);} int main () {   pid_t pid;    int count, SHMFD, ret;    int *shm_p;    sem = Mylock _init ();   &NBSP;IF (SEM = = NULL) {       fprintf (stderr, "Mylock_init (): error!\n"),     &NBSP ;  exit (1);    }   &NBSP;SHMFD = Shm_open (Shmpath, o_rdwr| O_creat| O_trunc, 0600);    if (SHMFD < 0) {       perror ("Shm_open ()"),        exit (1); &nbs P  }    ret = Ftruncate (shmfd, sizeof (int));   &NBSP;IF (Ret < 0) {       perror ("Ftruncate ()"),        exit (1); &nbsp ;  }    shm_p = (int *) mmap (NULL, sizeof (int), prot_write| Prot_read, map_shared, SHMFD, 0);    if (map_failed = = shm_p) {       perror ("mmap ()"),       &NBSP;EXIt (1);    }    *shm_p = 0;     for (count=0;count<count;count++) {       pid = fork (),       &NBSP;IF (PID < 0) {           perror ("fork ()"),            exit (1),   &NBS P   &NBSP;}        if (PID = = 0) {           do_child (shmpath); &nbs P      }    }    for (count=0;count<count;count++) {       wait (N ULL);    }    printf ("Shm_p:%d\n", *shm_p);    munmap (shm_p, sizeof (int));    close (SHMFD);    shm_unlink (Shmpath);    sleep (3000);    mylock_destroy (SEM);    exit (0);}

Anonymous semaphore usage:

[[email protected] sem]$ cat racing_posix_shm_unname.c#include <unistd.h> #include <stdlib.h># Include <stdio.h> #include <errno.h> #include <fcntl.h> #include <string.h> #include <sys/ file.h> #include <wait.h> #include <sys/mman.h> #include <sys/stat.h> #include <semaphore.h > #define COUNT 100#define shmpath "/shm" Static sem_t *sem;void mylock_init (void) {   sem_init (SEM, 1, 1);} void Mylock_destroy (sem_t *sem) {   sem_destroy (SEM);} int Mylock (sem_t *sem) {   while (sem_wait (SEM) < 0) {       if (errno = eintr) {&nbsp ;          continue;        }        perror ("sem_wait ()");        return-1;    }    return 0;} int Myunlock (sem_t *sem) {   if (sem_post (SEM) < 0) {       perror ("Semop ()");   &N Bsp    return-1;   &NBSP;}} int Do_child (char * shmpath) {   int interval, SHMFD, ret;    int *shm_p;   &NBSP;SHMFD = Shm_ope N (Shmpath, O_RDWR, 0600);    if (SHMFD < 0) {       perror ("Shm_open ()"),        exit (1); &nbs P  }    shm_p = (int *) mmap (NULL, sizeof (int), prot_write| Prot_read, map_shared, SHMFD, 0);    if (map_failed = = shm_p) {       perror ("mmap ()"),        exit (1); nbsp  }    /* critical section */   mylock (SEM);    interval = *shm_p;    interval++;    usleep (1);    *shm_p = interval;    myunlock (SEM);    /* critical section */   munmap (shm_p, sizeof (int));    close (SHMFD);    exit (0);}  int main () {   pid_t pid;    int count, SHMFD, ret;    int *shm_p;    sem = (sem_t *) mmap (NULL, sizeof (sem_t), Prot_write| Prot_read, map_shared| Map_anonymous,-1, 0);    if (void *) SEM = = map_failed) {       perror ("mmap ()"),        exit (1);    }    mylock_init ();   &NBSP;SHMFD = Shm_open (Shmpath, o_rdwr| O_creat| O_trunc, 0600);    if (SHMFD < 0) {       perror ("Shm_open ()"),        exit (1); &nbs P  }    ret = Ftruncate (shmfd, sizeof (int));   &NBSP;IF (Ret < 0) {       perror ("Ftruncate ()"),        exit (1); &nbsp ;  }    shm_p = (int *) mmap (NULL, sizeof (int), prot_write| Prot_read, map_shared, SHMFD, 0);    if (map_failed = = shm_p) {       perror ("mmap ()"),        exit (1); nbsp  }    *shm_p = 0;     for (count=0;count<count;count++) {       pid = fork (),       &NBSP;IF (PID < 0) {           perror ("fork ()"),            exit (1);       &NBSP,}        if (PID = = 0) {           do_child (Shmpath );        }    }    for (count=0;count<count;count++) {      &NBSP ; Wait (NULL);    }    printf ("Shm_p:%d\n", *shm_p);    munmap (shm_p, sizeof (int));    close (SHMFD);    shm_unlink (Shmpath);    sleep (3000);    mylock_destroy (SEM);    exit (0);}

The above procedure is not carefully studied, but simply lists the usage. It is also important to note that these programs need to be compiled with additional compilation parameters-LRT and-lpthread.

At last

Hopefully this will get you a better understanding of how much Linux is signaling. If you have any questions, you can contact me on my Weibo or blog.

Please scan the QR code, thank you for your donation!

Hello, I'm zorro!.

If you like this article, welcome to the Weibo search "Orroz" Follow me, address is: Http://weibo.com/orroz

You can also search on:Linux system technology follow my public number.

All my articles will be deposited on my personal blog, the address is: Http://liwei.life.

Welcome to use the above various ways to explore learning, common progress.

Public number QR Code:

"Reprint" of inter-process communication between Linux-semaphores