Deep understanding of Linux interprocess communication __linux

deep understanding of Linux interprocess communication (IPC)

0. Preface

1. Pipeline

1.1. Pipeline overview and related API application

1.2. Well-known pipeline overview and related API application

1.3. Summary

1.4. Reference materials

2. Signal (ON)

2.1. Signal and Signal source

2.2. Type of signal

2.3. Process response to the signal

2.4. Signal sending

2.5. Signal installation (set signal associated action)

2.6. Signal set and signal set operation function

2.7. Signal blocking and signal pending

2.8. Reference materials

3. Signal (Next)

3.1. Signal life cycle

3.2. Signal Programming considerations

3.3. Simple: Signal Application examples

3.4. Concluding remarks

3.5. Reference materials

4. Message Queuing

4.1. Message Queuing Basic Concepts

4.2. Operation Message Queuing

4.3. Message Queuing restrictions

4.4. Message Queuing Application Instance

4.5. Summary

4.6. Reference materials

5. Signal Light

5.1. Signal Lamp Overview

5.2. Linux semaphore

5.3. Signal and kernel

5.4. Operation Signal

5.5. Restrictions on traffic lights

5.6. Competition issues

5.7. Signal Lamp Application Example

5.8. Reference materials

6. Shared Memory (ON)

6.1. How the kernel ensures that each process addresses the memory pages of the same shared memory area

6.2. Mmap () and its associated system calls

6.3. Mmap () example

6.4. Access to Mmap () return address

6.5. Reference materials

7. Shared Memory (bottom)

7.1. System V Shared Memory principle

7.2. System V Shared Memory API

7.3. System V Shared memory limit

7.4. System V Shared Memory example

7.5. Conclusion

7.6. Reference materials

8. Socket interface

8.1. Background knowledge

8.2. Important Data structure

8.3. Several important steps in the programming of socket interface

8.4. Typical calling code

8.5. Other important concepts in network programming

8.6. Reference materials

This article is a series of articles organized from Network http://www.ibm.com/developerworks/cn/linux/l-ipc/.

Zheng Yanxin, National Institute of Computer Science, 0.

The process communication method under Linux is basically inherited from the process communication means on UNIX platform. The two main forces that make a significant contribution to the development of Unix are the at&t of Bell Labs and the BSD (Berkeley Software Publishing Center at UC Berkeley) with different emphasis on interprocess communication. The former has systematically improved and expanded the early interprocess communication means of UNIX, formed "System V IPC", the communication process is confined to a single computer, the latter has skipped the limit, and formed the interprocess communication mechanism based on socket (socket). Linux inherits both, as shown here:

Among them, the original UNIX IPC includes: pipeline, FIFO, signal; System V IPC includes: System V Message Queuing, System V Semaphore, System V shared memory area, POSIX IPC includes POSIX Message Queuing, POSIX semaphore, POSIX shared memory area. There are two points to be brief: 1 due to the diversity of UNIX versions, the Institute of Electronic and Electrical Engineering (IEEE) has developed an independent UNIX standard, the new ANSI Unix standard known as the Portable Operating System interface (Psoix) of the computer environment. Most of the existing UNIX and popular versions follow the POSIX standard, while Linux follows the POSIX standard from the outset; 2 BSD is not not involved in interprocess communication within a single machine (the socket itself can be used for interprocess communication within a single machine). In fact, many UNIX versions of stand-alone IPC are left with BSD traces, such as 4.4BSD-supported anonymous memory mappings, 4.3+BSD implementations of reliable signal semantics, and so on.

Figure I gives the various IPC tools supported by Linux, and in the next discussion in this article, in order to avoid conceptual confusion, the discussion of all the issues ultimately boils down to interprocess communication in the Linux environment, with as few references to UNIX versions as possible. And, for different implementations of Linux-supported communications, such as the POSIX shared memory area and the System V shared memory area of two implementations for shared memory, the POSIX API will be introduced primarily.

Several main means of interprocess communication under Linux Introduction: Pipelines (Pipe) and well-known pipelines (named Pipe): Pipelines can be used for communication between relational processes, and a well-known pipeline overcomes the restriction of a pipe without a name, so that, in addition to the functionality that the pipe has, It also allows communication between unrelated processes; signal (Signal): A signal is a more complex form of communication used to inform the receiving process that an event occurs, in addition to communication between processes, the process can send signals to the process itself; Linux in addition to supporting the UNIX early signal semantic function Sigal, It also supports the signal function sigaction of semantics conforming to POSIX.1 standard (in fact, the function is based on BSD, BSD in order to achieve reliable signal mechanism, but also can unify the external interface, with the Sigaction function to realize the signal function); Message queue (Message Queuing): The messages queue is a linked table of messages, including POSIX Message Queuing system V Message Queuing. A process with sufficient permissions can add messages to the queue, and processes that are given Read permission can read messages in the queue. Message Queuing overcomes the lack of information load, the pipeline can only host unformatted byte streams and buffer size limitations. Shared memory: Enables multiple processes to access the same memory space, which is the fastest available form of IPC. is designed for the low efficiency of other communication mechanisms. It is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and mutual exclusion between processes. Semaphore (semaphore): primarily as a means of synchronization between processes and between different threads of the same process. Socket: A more general interprocess communication mechanism that can be used for interprocess communication between different machines. Originally developed by the BSD branch of the UNIX system, it is now generally possible to migrate to other Unix-like systems: Linux and System V variants support sockets.

The above communication mechanism will be described in detail below.

Appendix 1: Reference [2] provides a general description of the processes in the Linux environment:

In general, the process under Linux consists of several key elements: an executable program, a dedicated system stack space, a control block (Process Control block) in the kernel, and a description of the resources occupied by the process, so that processes can accept the kernel's scheduling; have separate storage space

Processes and threads are sometimes not completely differentiated and often understand their meaning according to context.

reference materials UNIX Environment Advanced Programming, Author: W.richard Stevens, translator: Yu Jinyuan, Machinery Industry press. With rich programming examples, and key functions along with the development of UNIX process. Linux Kernel source code scenario analysis (up and down), Maudeca, Hu Himing, Zhejiang University Press, provides a very good analysis of the Linux kernel, while the background of some key concepts are described in detail. UNIX Network Programming Volume II: Inter-Process communication, Author: W.richard Stevens, translator: Yang Jizhang, Tsinghua University Press. A more comprehensive description of the communication between processes in a UNIX environment (no signal and socket interface, set interface in the first volume). 1. Pipeline

In this series, the author outlines several key means of communication between Linux processes. Among them, pipelines and well-known pipelines are one of the earliest interprocess communication mechanisms, pipelines can be used for communication between relational processes, and a well-known pipeline overcomes the limitations of a pipe without a name, so it allows communication between unrelated processes in addition to the functionality that the pipeline has. Recognizing the rules for reading and writing pipes and well-known pipelines is the key to applying them in a program, on the basis of discussing the communication mechanism of pipelines and famous pipelines in detail, this paper verifies the rules of reading and writing with examples, which is helpful to enhance readers ' perceptual knowledge of reading and writing rules, and also provides examples of application. 1.1. Pipeline Overview and related API application key concepts related to 1.1.1 pipeline

The pipeline is one of the original UNIX forms of IPC supported by Linux, with the following characteristics: The pipe is Half-duplex, the data flows only in one direction, and when both sides are required to communicate, two pipes need to be established; only for parent-child processes or sibling processes (relational processes); A separate file system: The pipeline is a file for the process at both ends of the pipe, but it is not a common file, it does not belong to a file system, it is a stand-alone file system, and exists only in memory. Read and write data: a process that writes to the pipe is read by the process on the other side of the pipe. The written content is added to the end of the pipe buffer each time, and data is read from the head of the buffer each time. creation of 1.1.2 pipelines

#include <unistd.h> int pipe (int fd[2])

The two ends of the pipe created by the function are in the middle of a process, it doesn't make much sense in practice, so a process typically fork a subprocess after a pipeline is created by pipe (), and then communicates between parent and child processes through pipelines (so it's easy to roll out, as long as there are relationships in two processes, The kinship here refers to having a common ancestor, which can be communicated by way of piping. Read and write rules for 1.1.3 pipelines

Each end of the pipe can be described by the description Word fd[0] and fd[1], and it should be noted that both ends of the pipe are fixed. That one end can only be used for reading, represented by the descriptive word fd[0], is called the pipe read end, and the other end can only be used for writing, represented by the descriptive word fd[1], which is called the pipe write end. Attempting to read data from a pipe write end, or writing data to a pipe-reading end, will cause an error to occur. General file I/O functions can be used in pipelines, such as close, read, write, and so on.

Reading data from a pipe: If the write end of the pipe does not exist, is considered to have read the end of the data, the Read function returns the number of read bytes 0; When the write end of the pipe is present, if the number of bytes requested is greater than pipe_buf, the number of data bytes existing in the pipeline is returned, if the number of bytes requested is not greater than pipe_ BUF, returns the number of existing data bytes in the pipeline (at which point the amount of data in the pipeline is less than the requested amount of data), or returns the number of bytes requested (at which point the amount of data in the pipeline is not less than the requested amount of data). Note: (Pipe_buf is defined in Include/linux/limits.h, different kernel versions may vary.) Posix.1 requires a pipe_buf of at least 512 bytes and Red Hat 7.2 as 4096).

About read rule validation for pipelines:

/************** * READTEST.C * **************/ #include <unistd.h> #include <sys/types.h> #include <errno.h> Main () { int pipe_fd[2]; pid_t pid; Char r_buf[100]; Char w_buf[4]; char* P_wbuf; int r_num; int cmd; memset (r_buf,0,sizeof (R_BUF)); memset (w_buf,0,sizeof (R_BUF)); P_wbuf=w_buf; if (pipe (PIPE_FD) <0) { printf ("Pipe Create error\n"); return-1; } if ((Pid=fork ()) ==0) { printf ("\ n"); Close (pipe_fd[1]); Sleep (3);//Ensure that the parent process closes the write end R_num=read (pipe_fd[0],r_buf,100); printf ("Read num is%d the data read from the pipe is%d\n", R_num,atoi (R_BUF)); Close (pipe_fd[0]); Exit (); } else if (pid>0) { Close (pipe_fd[0]);//read strcpy (W_buf, "111"); if (write (pipe_fd[1],w_buf,4)!=-1) printf ("Parent write over\n"); Close (pipe_fd[1]);//write printf ("Parent close fd[1] over\n"); Sleep (10); } } /************************************************** * Program Output results: * Parent Write over * Parent Close fd[1] Over * Read NUM is 4 the data read to the pipe is 111 * Additional conclusions: * When the pipe end is closed, the data written will remain until it is read. ****************************************************/

Writing data to a pipeline: when writing data to a pipeline, Linux will not guarantee the atomic nature of the write, and if the pipeline buffer has an empty area, the write process will attempt to write data to the pipe. If the read process does not read the data in the pipeline buffer, the write operation will remain blocked.
Note: It makes sense to write data to a pipe only when the end of the pipe is read. Otherwise, the process that writes data to the pipeline receives the sifpipe signal from the kernel, which the application can process or ignore (the default action is the application terminates).

Validation of write rules for pipelines 1: The dependence of the writing end on the read end

#include <unistd.h> #include <sys/types.h> Main () { int pipe_fd[2]; pid_t pid; Char r_buf[4]; char* W_buf; int writenum; int cmd; memset (r_buf,0,sizeof (R_BUF)); if (pipe (PIPE_FD) <0) { printf ("Pipe Create error\n"); return-1; } if ((Pid=fork ()) ==0) { Close (pipe_fd[0]); Close (pipe_fd[1]); Sleep (10); Exit (); } else if (pid>0) { Sleep (1); Wait for child process to finish closing the read-side operation Close (pipe_fd[0]);//write W_buf= "111"; if ((Writenum=write (pipe_fd[1],w_buf,4)) ==-1) printf ("Write to Pipe error\n"); Else printf ("The bytes write to pipe is%d \ n", writenum); Close (pipe_fd[1]); } }

The output is: Brokenpipe, because the end of the pipe and all of its fork () products have been closed. If you keep the read end in the parent process, that is, after you finish writing the pipe, and then close the parent process's read end, it will also write to pipe, the reader can verify the conclusion. Therefore, when writing data to a pipe, at least one process should exist in which the pipe read end is not closed, or the above error occurs (the pipeline breaks, the process receives the sigpipe signal, and the default action is the process terminates)

Validation of write rules for pipelines 2:linux does not guarantee atomic verification of write pipelines

#include <unistd.h> #include <sys/types.h> #include <errno.h> Main (int argc,char**argv) { int pipe_fd[2]; pid_t pid; Char r_buf[4096]; Char w_buf[4096*2]; int writenum; int rnum; memset (r_buf,0,sizeof (R_BUF)); if (pipe (PIPE_FD) <0) { printf ("Pipe Create error\n"); return-1; } if ((Pid=fork ()) ==0) { Close (pipe_fd[1]); while (1) { Sleep (1); Rnum=read (pipe_fd[0],r_buf,1000); printf ("Child:readnum is%d\n", rnum); } Close (pipe_fd[0]); Exit (); } else if (pid>0) { Close (pipe_fd[0]);//write memset (r_buf,0,sizeof (R_BUF)); if ((Writenum=write (pipe_fd[1],w_buf,1024)) ==-1) printf ("Write to Pipe error\n"); Else printf ("The bytes write to pipe is%d \ n", writenum); Writenum=write (pipe_fd[1],w_buf,4096); Close (pipe_fd[1]); } } Output results: The bytes write to pipe 1000 The bytes write to pipe 1000//Note that the output of this line illustrates the non-atomicity of the Write The bytes write to pipe 1000 The bytes write to pipe 1000 The bytes write to pipe 1000 The bytes write to pipe 120//Note that the output of this line illustrates the non-atomicity of the Write The bytes write to pipe 0 The bytes write to pipe 0 ......

Conclusion:

Writes to a non atom when the number of writes is less than 4096.
If you change the number of two write bytes in the parent process to 5000, it is easy to draw the following conclusion:
When the amount of data written to the pipe is greater than 4096 bytes, the free space of the buffer is written to the data (padded) until all the data has been written, and if no process reads the data, it blocks. 1.1.4 Pipeline Application Example

Example one: for Shell

A pipe can be used for input-output redirection, which directs the output of one command directly to the input of another command. For example, when you type who│wc-l in a shell program (Bourneshell or C shell, etc.), the shell program creates the WHO and the WC two processes and the pipelines between the two processes. Consider the following command line:

$kill-L operation results are attached to one.