Inter-process communication-pipelines-Linux Kernel Analysis (8)

Inter-process communication-pipelines-Linux Kernel Analysis (8)
MPs queue

A pipe is a one-way communication mechanism between two processes.

Because pipelines transmit data in one direction, pipelines are also called half-duplex pipelines.

This feature of MPs queues determines the limitations of the tool. As one of the first Unix IPC formats supported by Linux, pipelines have the following features:

Data can only flow from one process to another (one read pipeline and one write pipeline). To perform duplex communication, you need to establish two pipelines.

The MPs queue can only be used for communication between parent and child processes or brokers ., That is to say, the MPs queue can only be used for Kinship-related inter-process communication.

In addition to the preceding limitations, the MPs queue has other shortcomings, such as the MPs queue with no name (anonymous MPs Queue) and the buffer size of the MPs queue is limited. A media transcoding queue transmits a non-formatted byte stream. In this case, the data format needs to be agreed between the input and output parties in advance. Although there are so many deficiencies, pipelines are fully competent for some simple inter-process communication.

Differences between signals and messages

We know that the signal communication mechanism between processes is based on the signal when transmitting information, but the information carrier of the pipeline communication mechanism is the message. So what is the difference between a signal and a message?

First of all, in terms of data content, the signal is just some pre-defined code, used to indicate a certain situation of the system; the message is a set of continuous statements or symbols, but the amount is not too large. In terms of function, the signal serves as the transmission of a small amount of information between processes. Generally, the kernel program is used to notify user processes of some exceptions, and the message is used to exchange data between processes.

In terms of sending time, a signal can be sent at any time; information cannot be sent at any time. In terms of the sender, the signal cannot determine who the sender is; Information knows who the sender is. In terms of sending objects, a signal is sent to a process, while a message is sent to a message queue. In terms of processing methods, the signal can be ignored; the message must be processed. In terms of data transmission efficiency, the signal is not suitable for a large amount of information transmission because of its low efficiency. Although the message is not suitable for a large amount of data transmission, it is more efficient than the signal, therefore, it is suitable for medium data transmission.

MPs queue-differences between named MPs queues

We know that both named pipelines and pipelines can send messages between processes, but they are also different.

Pipeline communication methods have two restrictions,

One is half-duplex communication, and data can only flow in one direction

Second, it can only be used between unrelated processes.

The kinship of a process usually refers to the parent-child process relationship.

Stream pipelines_pipeRemove the first restriction, which can be transmitted in two directions.

Pipes can be used for communications between kinship-related processes, Named Pipesname_pipeThe pipe has no name restrictions. Therefore, in addition to the functions of the pipe, it also allows communication between unrelated processes;

Pipeline technology can only be used to connect processes with common ancestor, such as the communication between parent and child processes. It cannot share information between processes of different users. In addition, the MPs queue cannot be established permanently. When the process for accessing the MPs queue is terminated, the MPs queue is also withdrawn. These restrictions impose many restrictions on its use, but the named pipe has overcome these restrictions.

A named pipe, also known as FIFO, is a permanent mechanism. A fifo file also has attributes such as file name, file length, and access permission. It can be opened, closed, and deleted as other Linux Files, so any process can find it. In other words, even different ancestor processes can use Named Pipes for communication.

If you want full-duplex communication, you 'd better use the Sockets API (linux does not support s_pipe Stream pipelines ). Next we will introduce the technical model of pipeline parsing through pipeline commands, and then detail the programming interfaces and system-level commands used for pipeline programming.

Pipeline technical Model

Pipeline Technology is a long-standing inter-process communication mechanism in Linux.

All pipeline technologies, whether half-duplex anonymous pipelines or named pipelines, use the FIFO Queuing Model to direct inter-process communication.

For pipelines, we can regard them as a unidirectional connector connecting two entities visually.

When a media transcoding queue is used for communication, the processes at both ends read and write data to the media transcoding queue through the file descriptor set by the system when the media transcoding queue is created. In essence, an MPS queue is also a file, but it is different from a common file. It can overcome the two problems of file communication. This file only exists in the memory.

One process writes data to the pipeline and the other reads data from the pipeline. Each time the written data is added to the end of the MPs queue buffer, data is read from the buffer header during data reading.

Pipeline command details

See

Linux shell pipeline command (pipe) usage and shell redirection difference

For example, see the following command

Pipeline symbol is a powerful unix feature. It is a vertical line: "| ",

Usage: command 1 | command 2

Its function is to set the first commandcommand 1Execution resultcommand 2Inputcommand 2

Note:

Pipeline commands only process the correct output of the previous command, without handling the error output

The command on the right of the pipeline command must be able to receive standard input stream commands.

For example:
ls -l | more

This command lists any documents in the current directory and sends the output to the more command as the input. The more command displays the file list on pages.

Differences between pipeline commands and redirection

The difference is:

Standard output should be provided for the left-side command | standard input should be accepted for the right-side command.

The command on the left should have standard output> only files on the right

Standard input is required for the command on the left <only files on the right

Pipeline triggers two sub-processes to execute "|" programs on both sides, and redirection is executed in one process.

Redirection and pipelines can be used in many cases

In shell, it is oftenGreat roads to Rome.

Generally, it is better to use the redirection output if the parameters are passed between commands or the pipeline is good. If the output result needs to be redirected to a file, it is better to use the redirection output.

In the preceding example, an pipeline is established between two commands (sometimes called a command pipeline operation ).

The output produced after the first command ls is used as the input of the second command more.

This is a half-duplex communication because the communication is unidirectional. The specific work of connecting two commands is completed by the kernel.

Of course, the kernel also provides us with a set of interfaces (system calls). In addition to commands, applications can also use pipelines to connect.

Pipeline programming technical pipeline Interface Unknown pipeline pipe

Create pipeline pipe

Function prototype 'int pipe (int filedes [2]);

pipe()Creates an MPS queue and adds the file description by parameters.filedesArray.

filedes[0]It is the read end in the pipeline, So pipe uses read to call.

filedes[1]It is the write end of the MPs queue. Use write to write data.

Return Value

If it succeeds, zero is returned. Otherwise,-1 is returned. The error cause is stored in errno.

Error Code

The EMFILE process has used up the maximum number of file descriptions

The ENFILE system does not have a file description word available.

The filedes array address of the EFAULT parameter is invalid.

When the call is successful, the pipe function returns 0; otherwise, the value is-1. When a successful response is returned, the array fds is filled with two valid file descriptors. The file descriptor in the first element of the array is supplied for program reading. The file descriptor in the second element of the array can be used for program writing.

Close Pipe

To close the pipeline, you only need to close the two file descriptors. You can use the common close function to close the file one by one.

If the write end of the MPs queue is closed, but some processes attempt to read data from the MPs queue, 0 is returned to indicate that the MPs queue is unavailable and should be closed. If the Reading end of the MPs queue is closed, but other processes attempt to write data to the MPs queue, the process attempting to write data will receive a SIGPIPE signal, the specific signal processing depends on the signal processing program.

Dup and dup2 Functions

Dup and dup2 are two very useful calls. They are used to copy the descriptor of a file.

They are often used to redirect stdin, stdout, and stderr of processes.

The two functions are prototype as follows:

#include 
  
   int dup( int oldfd );int dup2( int oldfd, int targetfd )
  

Dup Function

Using the dup function, we can copy a descriptor. If you pass it to the function an existing descriptor, it will return a new descriptor, which is a copy of the descriptor sent to it. This means that the two descriptors share the same data structure.

For example, if we perform the lseek operation on a file descriptor, the location of the first file is the same as that of the Second file. The following code snippet describes how to use the dup function:

int fd1, fd2;fd2 = dup( fd1 );

Note that we can create a descriptor before calling fork, which is the same as calling dup to create a descriptor. The sub-process will also receive a copied descriptor.

Dup2 Functions

The dup2 function is similar to the dup function, but the dup2 function allows the caller to specify the id of a valid Descriptor and a target descriptor.

When the dup2 function returns a successful result, the target Descriptor (the second parameter of the dup2 function) is changed to a replica of the source Descriptor (the first parameter of the dup2 function). In other words, the two file descriptors now point to the same file and are the files pointed to by the first parameter of the function.

The following code is used to describe:

int oldfd;oldfd = open("app_log", (O_RDWR | O_CREATE), 0644 );dup2( oldfd, 1 );close( oldfd );

We opened a new file called "app_log" and received a file descriptor called fd1. We call the dup2 function with the oldfd and 1 parameters. This will cause the file descriptor represented by 1 to be replaced with the newly opened file descriptor (that is, stdout, because the id of the standard output file is 1 ). Everything written to stdout is now written into the file named "app_log. It should be noted that after the dup2 function copies the oldfd, it will immediately close it, but will not turn off the newly opened file descriptor, because file descriptor 1 now points to it.

Named Pipe mkfifo

The mkfifo function is used to create a file in the file system. This file is used to provide the FIFO function, that is, the named pipe. The pipelines mentioned above have no names, so they are called anonymous pipelines or pipelines for short. For file systems, the anonymous pipeline is invisible, and its role is limited to communication between the parent process and the child process. The named pipe is a visible file. Therefore, it can be used for communication between any two processes, regardless of whether the two processes are Parent and Child processes or whether they are related. The following is a prototype of the Mkfifo function:

#include 
  
   #include 
   
    int mkfifo( const char *pathname, mode_t mode );
   
  

The mkfifo function requires two parameters. The first parameter (pathname) is a private file to be created in the file system. The second parameter (mode) is used to specify the read/write permissions of the FIFO. If the Mkfifo function is successfully called, the return value is 0. If the call fails, the return value is-1. The following example shows how to use the mkfifo function to create a fifo instance. The Code is as follows:

Int ret; ret = mkfifo ("/tmp/cmd_pipe", S_IFIFO | 0666); if (ret = 0) {// The named pipe is successfully created} else {// An error occurred while creating the named pipe}

In this example, a named pipe (fifo) is created using the pipeline _pipe file in the/tmp directory ). Then, you can open the file to perform read and write operations and communicate with each other. Once the named pipe is opened, you can use a typical input/output function to read content from it. For example, the following code segment shows how to read content from the pipeline using the fgets function:

    pfp = fopen( "/tmp/cmd_pipe", "r" );    ret = fgets( buffer, MAX_LINE, pfp );

We can also write content to the pipeline. The following code segment shows you how to write content to the pipeline using the fprintf function:

    pfp = fopen( "/tmp/cmd_pipe", "w+ );    ret = fprintf( pfp, "Here’s a test string!\n" );

For a named pipe, the reader cannot open the named pipe unless the writer actively opens the Reading end of the pipe. After the Open call is executed, the reader is locked until the writer appears. Although named pipes have such limitations, they are still an effective tool for inter-process communication.

Unknown MPs queue

The Nameless pipeline provides a communication pipeline for the process of establishing the pipeline and its descendants to transmit messages in bit streams.

The media transcoding queue is logically regarded as a media transcoding queue file, which is physically composed of a file system's high-speed buffer, and rarely starts peripherals.

The sending process uses the system call of the file systemwrite(fd[1],buf,size)bufThe length issizeThe character message is sent to the pipeline entry.fd[1],

The receiving process is called by the system.read(fd[0],buf,size)Export from MPs queuefd[0]Exit readingsizeCharacter message placementbuf.

Here, the pipeline sends messages in FIFO (first-in-first-out) mode, and only one-way message transmission () is allowed ().

Anonymous pipeline pipe read/write

Pipelines are used for communication between different processes. Create an MPS queue first and then create a sub-process through the fork function. The sub-process inherits the MPs queue created by the parent process. Note: pipe () must be called before fork () is called. Otherwise, the sub-process will not inherit the file descriptor. Otherwise, two pipelines are created. Because the Parent and Child processes share the same code segment, pipe () is called to create two pipelines, and an exception error occurs.

Read/write process of unknown Pipeline

#include 
  
   #include 
   
    #include 
    
     #include 
     
      #include 
      
       #include 
       
        #define MAX_DATA_LEN 256#define DELAY_TIME 1int main(void){ pid_t pid; char buf[MAX_DATA_LEN]; const char *data="Pipe Test program"; int real_read, real_write; int pipe_fd[2]; memset((void*)buf, 0, sizeof(buf)); if(pipe(pipe_fd) < 0) { perror("Pipe create error...\n"); exit(1); } else { printf("Pipe create success...\n"); } if ((pid = fork()) < 0) { perror("Fork error!\n"); exit(1); } else if (pid == 0) { printf("I am the child process, PID = %d, PPID = %d", getpid(), getppid()); close(pipe_fd[1]); sleep(DELAY_TIME * 3); if ((real_read=read(pipe_fd[0],buf, MAX_DATA_LEN)) > 0) { printf("Child receive %d bytes from pipe: '%s'.\n", real_read, buf); } close(pipe_fd[0]); exit(0); } else { printf("I am the parent process, PID = %d, PPID = %d", getpid(), getppid()); close(pipe_fd[0]); sleep(DELAY_TIME); if ((real_write = write(pipe_fd[1], data, strlen(data))) > 0) { printf("Parent write %d bytes into pipe: '%s'.\n", real_write, data); } close(pipe_fd[1]); waitpid(pid,NULL,0); exit(0); } return EXIT_SUCCESS;}
       
      
     
    
   
  

Multi-process pipeline read/write

Create an MPS queue. At the same time, the parent process generates child processes P1 and P2, which respectively write their own strings to the pipeline, and the parent process reads them ().

# Include <stdio. h> main () {int I, r, p1, p2, fd [2]; char buf [50], s [50]; pipe (fd ); /* the parent process creates a pipeline */while (p1 = fork () =-1); if (p1 = 0) {lockf (fd [1], 1, 0);/* lock the write end */sprinrf (buf, "child process P1 is sending messages! \ N "); printf (" child process P1! \ N "); write (fd [1], buf, 50);/* write 50 characters in the buf to the MPs queue */sleep (5 ); lockf (fd [1],);/* release the MPs queue write end */exit (0);/* close P1 */} else/* return from the parent process, execute the parent process */{while (p2 = fork () =-1);/* Create the child process P2, loop when the failure */if (p2 = 0) /* return from the child process P2, execute P2 */{lockf (fd [1],);/* lock the write end */sprintf (buf, "child process P2 is sending messages \ n"); printf ("child process P2! \ N "); write (fd [1], buf, 50);/* write the characters in the buf to the MPs queue */sleep (5 ); /* sleep wait */lockf (fd [1],);/* release the MPs queue write end */exit (0 ); /* disable P2 */} wait (0); if (r = read (fd [0], s 50) =-1) printf ("can't read pipe \ n"); else printf ("% s \ n", s); wait (0 ); if (r = read (fd [0], s, 50) =-1) printf ("can't read pipe \ n "); else printf ("% s \ n", s); exit (0 );}}

Use the dup function to implement Command Flow

Our child process redirects its output to the input of the pipeline, and then the parent process redirects its input to the output of the pipeline. This is a very useful technology in actual application development.

#include 
  
   #include 
   
    #include 
    
     int main(){    int pfds[2];    if ( pipe(pfds) == 0 )    {        if ( fork() == 0 )        {            close(1);            dup2( pfds[1], 1 );            close( pfds[0] );            execlp( "ls", "ls", "-1", NULL );        }        else        {            close(0);            dup2( pfds[0], 0 );            close( pfds[1] );            execlp( "wc", "wc", "-l", NULL );        }    }    return 0;}
    
   
  

Named Pipe Write end

# Include
  
   
# Include
   
    
# Include
    
     
# Include
     
      
# Include
      
        # Include
       
         # Include
        
          # Include
         
           # Define FIFO "myfifo" # define BUFF_SIZE 1024int main (int argc, char * argv []) {char buff [BUFF_SIZE]; int real_write; int fd; if (argc <= 1) {printf ("Usage: % s string \ n", argv [0]); exit (1 );} else {printf ("% s at PID = % d \ n", argv [0], getpid ();} sscanf (argv [1], "% s ", buff); // test whether the FIFO exists. if it does not exist, mkfifo is a FIFO if (access (FIFO, F_ OK) =-1) {if (mkfifo (FIFO, 0666) <0) & (errno! = EEXIST) {printf ("Can NOT create fifo file! \ N "); exit (1) ;}// call open to open FIFO in write-only mode, and return the file descriptor fd if (fd = open (FIFO, O_WRONLY )) =-1) {printf ("Open fifo error! \ N "); exit (1) ;}// call write to write the buff to the first-in-first-out (if (real_write = write (fd, buff, BUFF_SIZE) pointed by the file descriptor fd> 0) {printf ("Write into pipe: '% s '. \ n ", buff); exit (1);} close (fd); exit (0 );}
         
        
       
      
     
    
   
  

Read end

# Include
  
   
# Include
   
    
# Include
    
     
# Include
     
      
# Include
      
        # Include
       
         # Include
        
          # Include
         
           # Define FIFO "myfifo" # define BUFF_SIZE 1024int main (int argc, char * argv []) {char buff [BUFF_SIZE]; int real_read; int fd; printf ("% s at PID = % d", argv [0], getpid (); // access determines the file or folder access permission. That is, check the access method of a file. // if the specified access method is valid, the function returns 0; otherwise, the function returns-1. // if no FIFO exists, create an if (access (FIFO, F_ OK) =-1) {if (mkfifo (FIFO, 0666) <0) & (errno! = EEXIST) {printf ("Can NOT create fifo file! \ N "); exit (1) ;}// open the FIFO in read-only mode and return the file descriptor fd if (fd = open (FIFO, O_RDONLY )) =-1) {printf ("Open fifo error! \ N "); exit (1) ;}// call read to read the first-in-first-out content directed by fd to the buff, and print while (1) {memset (buff, 0, BUFF_SIZE); if (real_read = read (fd, buff, BUFF_SIZE)> 0) {printf ("Read from pipe: '% s '. \ n ", buff) ;}} close (fd); exit (0 );}