Implement two-way inter-process communication pipeline on Linux

Article Title: Implement two-way inter-process communication pipeline on Linux. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.
　　 Problems and existing methods
Linux provides popen and pclose functions for creating and disabling pipelines to communicate with another process. The interface is as follows:
　　
FILE * popen (const char * command, const char * mode );
Int pclose (FILE * stream );
　　
Unfortunately, the pipeline created by popen can only be one-way-the mode can only be "r" or "w", but not a combination-the user can only choose to either write in, you can either read from it, but cannot read or write in a single pipeline at the same time. In practical applications, there are often requirements for reading and writing at the same time. For example, we may want to send text data to the sort tool for sorting and then retrieve the results. In this case, the popen cannot be used. We need to find other solutions.
　　
One solution is to use the pipe function to create two unidirectional pipelines. The code without error detection is as follows:
　　
Int pipe_in [2], pipe_out [2];
Pid_t pid;
Pipe (& pipe_in); // creates an MTS queue for reading data in the parent process.
Pipe (& pipe_out); // creates an MPS queue for Data Writing in the parent process.
If (pid = fork () = 0) {// sub-process
Close (pipe_in [0]); // close the sub-process read end of the read pipeline of the parent process
Close (pipe_out [1]); // closes the child process write end of the write pipeline of the parent process
Dup2 (pipe_in [1], STDOUT_FILENO); // copy the read pipeline of the parent process to the standard output of the Child Process
Dup2 (pipe_out [0], STDIN_FILENO); // copy the standard input from the writing pipeline of the parent process to the child process
Close (pipe_in [1]); // close the read pipeline that has been copied
Close (pipe_out [0]); // close the copied write Pipeline
/* Execute the command using exec */
} Else {// parent process
Close (pipe_in [1]); // close the write end of the read Pipeline
Close (pipe_out [0]); // close the read end of the write Pipeline
/* You can write data to pipe_out [1] and read the result from pipe_in [0 */
Close (pipe_out [1]); // close the write Pipeline
/* Read the remaining data in pipe_in [0 */
Close (pipe_in [0]); // close the read Pipeline
/* Use wait functions to wait for the sub-process to exit and obtain the exit code */
}
　　
Of course, this code is less readable (especially after the error processing code is added) and cannot be encapsulated into functions similar to popen/pclose, making it easy for high-level code to use. The reason is that a pair of file descriptors returned by the pipe function can only be read from the first and written to the second (at least for Linux ). In order to read and write data at the same time, we can only use the cumbersome two pipe calls and two file descriptors.
　　
　　 New Method
This is the only way to use pipe. However, Linux implements a socketpair call from BSD, which can implement the Read and Write Functions in the same file descriptor (this call is currently part of the POSIX specification ). This system call can create a pair of connected (UNIX) unknown sockets. In Linux, this pair of sockets can be used as the file descriptor returned by pipe. The only difference is that any one of these file descriptors can be read and writable.
　　
This is close to what we want. However, there is still a problem that hinders us from using socketpair to communicate with a sub-process: in order to solve the problem of using sort, we need to disable the standard input of the sub-process to notify the sub-process that the data has been sent, and then read the data from the standard output of the sub-process until an EOF occurs. If two one-way pipelines are used, each pipeline can be closed independently, so this problem does not exist. When two-way pipelines are used, if the pipeline is not closed, the peer data cannot be notified that it has been sent, however, once the MPs queue is closed, the result data cannot be read from it. -- If this problem is not solved, the idea of using socketpair becomes meaningless.
　　
After searching and testing, I found that the shutdown call can solve this problem. After all, the file descriptor generated by socketpair is a pair of sockets, and standard operations on the socket can be used, including shutdown. -- Shutdown can be used to implement a half-shutdown operation, notifying the peer process not to send data, and still using this file descriptor to receive data from the peer end. The code without error detection is as follows:
　　
Int fd [2];
Pid_t pid;
Socketpair (AF_UNIX, SOCKET_STREAM, 0, fd); // create an MPS queue
If (pid = fork () = 0) {// sub-process
Close (fd [0]); // closes the parent process of the Pipeline
Dup2 (fd [1], STDOUT_FILENO); // copy the sub-process of the MPs queue to the standard output.
Dup2 (fd [1], STDIN_FILENO); // copy the sub-process of the MPs queue to the standard input.
Close (fd [1]); // close the read pipeline that has been copied
/* Execute the command using exec */
} Else {// parent process
Close (fd [1]); // closes the sub-process end of the Pipeline
/* Data can now be read and written in fd [0 */
Shutdown (fd [0], SHUT_WR); // notifies the peer that data has been sent
/* Read the remaining data */
Close (fd [0]); // close the MPs queue
/* Use wait functions to wait for the sub-process to exit and obtain the exit code */
}
　　
It is clear that this is much simpler than using two one-way pipelines. I will further encapsulate and improve it on this basis.
　　
　　 Encapsulation and implementation
Using the above method directly, no matter what you think, is at least ugly and inconvenient. The program maintainer wants to see the logic of the program, rather than the complicated details of completing a task. We need a good encapsulation.
　　
C or C ++ can be used for encapsulation. Here, I provide a C encapsulation similar to the popen/pclose function call in POSIX standards in a UNIX tradition to ensure maximum availability. The interface is as follows:
　　
FILE * dpopen (const char * command );
Int dpclose (FILE * stream );
Int dphalfclose (FILE * stream );
　　
Pay attention to the following points about interfaces:
　　
Similar to the pipe function, dpopen returns a pointer to the file structure rather than a file descriptor. This means that you can directly use functions such as fprintf. The File Buffer caches the data written to the pipeline (unless you disable the File Buffer using the setbuf function ), to ensure that the data is indeed written to the MPs queue, the fflush function is required.
　　
Because dpopen returns a read/write pipeline, the second parameter of popen that represents read/write is no longer needed.
　　
In a two-way pipeline, we need to notify the peer that data writing has ended. This operation is completed by the dphalfclose function.
　　
For specific implementation, please directly view the program source code, including detailed comments and doxygen documentation comments. I will only give a few notes:
　　
This implementation uses a linked list to record the correspondence between all the file pointers opened by dpopen and the sub-process IDs. Therefore, when there are many pipelines opened by dpopen at the same time, dpclose (you need to search for a linked list) is a little slower. In my opinion, this will not cause any problems during normal use. If this is a problem in some special cases, you can consider changing the return value type of dpopen and the input parameter type of dpclose (not easy to use, but easy to implement ), you can also use a hash table or a balance tree to replace the currently used linked list to accelerate search (the interface remains unchanged, but the implementation is complicated ).
　　
When the "-pthread" command line parameter is used in gcc during compilation, this implementation enables POSIX thread support and uses mutex to protect access to the linked list. Therefore, this implementation can be safely used in the POSIX multi-threaded environment.
　　
Similar to popen, dpopen closes pipelines previously opened with dpopen In The subprocesses generated by fork.
　　
If the parameter passed to dpclose is not a non-NULL value returned by dpopen, The errno is set to EBADF in addition to the returned-1 error. For pclose, this situation is considered unspecified in the POSIX specification.
　　
The implementation does not use any platform-related features to facilitate porting to other POSIX platforms.
　　
The following code shows a simple example of sending multiple lines of text to sort, then retrieving and displaying the results:
　　
# Include
# Include
# Include "dpopen. h"
　　
# Define MAXLINE 80
　　
Int main ()
{
Char line [MAXLINE];
FILE * fp;
Fp = dpopen ("sort ");
If (fp = NULL ){
Perror ("dpopen error ");
Exit (1 );
}
Fprintf (fp, "orange \ n ");
Fprintf (fp, "apple \ n ");
Fprintf (fp, "pear \ n ");
If (dphalfclose (fp) <0 ){
Perror ("dphalfclose error ");
Exit (1 );
}
For (;;){
If (fgets (line, MAXLINE, fp) = NULL)
Break;
Fputs (line, stdout );
}
Dpclose (fp );
Return 0;
}
　　
Output result:
　　
Apple
Orange
Pear
　　
　　 Summary
This article describes how to use the socketpair system call to implement a two-way process Communication Pipeline on Linux, and provides an implementation. The interface provided by this implementation is similar to the popen/pclose function in the POSIX specification, so it is very easy to use. This implementation does not use platform-related features, so it can be transplanted to the POSIX system that supports socketpair calls without modification or a few modifications.