Pipeline communication under Linux

Pipelines and message queues are essentially the same, all of which are buffers of the Linux kernel.

Pipe limits
The pipe is half-duplex, and the data can only flow in one direction; two pipelines are required to communicate with each other
Can only be used to communicate between processes that have a common ancestor (a process that has affinity); Typically, a pipeline is created by a process, and then the process calls fork, and the pipeline is then applied between the parent and child processes. The plumbing is getting smaller.

Pipe reading and writing rules

When there is no data to read

The read call is blocked, that is, the process pauses execution until the data arrives.

If the setting does not wait, the read call returns a -1,errno value of Eagain.

When the pipes are full

The write call blocks until a process reads the data.

If the setting does not wait, the call returns a -1,errno value of Eagain

When the pipes are written and full

Write Call blocking

The call returns a -1,errno value of Eagain

If the corresponding file descriptor for all the pipe writes is closed, read returns

The write operation generates a signal if the file descriptor corresponding to the read end of all the pipes is closed sigpipe

When the amount of data to be written is not greater than PIPE_BUF, Linux guarantees the atomicity of the write.

When the amount of data to be written is greater than Pipe_buf, Linux will no longer guarantee the atomicity of writes.

Pipeline essence, a fixed-size buffer in the kernel.

The command in the shell script is essentially implemented by pipeline technology.

A pipeline is created first, and then several commands are created to create several processes

For the process that executes the first command, the read end of the process is closed, and the write end of the marked pipe is copied to the standard output using the DUP2 command, and then the EXEC function family is executed, which is equivalent to writing the pipe. For a process that executes the second command,

The write end is closed and the read end of the pipe is copied to the standard output using the DUP2 command, and then the EXEC Function Group command is executed, so that the output of the first command is equivalent to the input of the second command, and the redundant commands are similar.

Named pipes

One limitation of pipeline applications is that they can only communicate between processes that have a common ancestor (affinity).

If we want to exchange data between unrelated processes, you can do this with a FIFO file, which is often referred to as a named pipe. A named pipe is a special type of file.

Create a named pipe

Named pipes can be created from the command line by using the following command: MKFIFO filename

Named pipes can also be created from the program, and related functions are: int mkfifo (const char *filename,mode_t);

Pipeline communication under Linux