Process Communication----pipeline (pipe)

Implementation mechanism of Linux pipelines

In Linux, pipelines are a very frequent communication mechanism. In essence, a pipeline is also a file, but it is different from the general file, the pipeline can overcome the use of files to communicate the two problems, in particular, the performance is:

· Limits the size of the pipe. In fact, a pipeline is a fixed-size buffer. In Linux, the buffer size is 1 pages, or 4K bytes, so that it does not grow as much as the file does. Using a single fixed buffer can also cause problems, such as when writing a pipeline, which can become full when this happens, then the write () call to the pipeline will be blocked by default, waiting for some data to be read so that there is enough space for write () to call.

· The read process may also work faster than the write process. When all current process data has been read, the pipeline becomes empty. When this happens, a subsequent read () call will be blocked by default, waiting for some data to be written, which resolves the issue where the read () call returns the end of the file.

Note : reading data from a pipe is a one-time operation, once the data is read, it is discarded from the pipeline, freeing up space to write more data.

1. Structure of the piping

In Linux, the implementation of pipelines does not use a dedicated data structure, but instead uses the file structure of the filesystem and the index node inode of the VFS. By pointing two file structures to the same temporary VFS index node, the VFS index node points to a physical page.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/84/51/wKiom1eMxlqCws7kAABk688oTZ8777.png "title=" wq.png "alt=" Wkiom1emxlqcws7kaabk688otz8777.png "/>

two file data structures, but they define files with a different action routine address, one of which is a routine address that writes data to the pipeline, and the other is a routine address that reads data from the pipeline. In this way, the system call of the user program is still the usual file operation, but the kernel uses this abstraction mechanism to realize the special operation of the pipeline.

2. Pipeline reading and writing

Pipeline implementation of the source code in FS/PIPE.C, there are many functions in the pipe.c, of which two functions are more important, namely the pipe reading function pipe_read () and the pipe write function Pipe_wrtie (). The pipe write function writes the data by copying the bytes to the physical memory pointed to by the VFS index node, while the pipe read function reads the data by copying the bytes in the physical memory. Of course, the kernel must use a mechanism to synchronize access to the pipe, and for this reason the kernel uses locks, waits for queues, and signals.

When the write process writes to the pipeline, it takes advantage of the standard library function write (), and the file structure of the file can be found by the system based on the document descriptor passed by the library function. The file structure specifies the address of the function (that is, the write function) that is used for the write operation, and the kernel calls the function to complete the write operation. Before the Write function writes data to memory, it must first check the information in the VFS index node and meet the following criteria for actual memory copy work:

• There is enough space in the memory to hold all the data to be written;

• The memory is not locked by the read program.

If the above conditions are met, the Write function first locks the memory and then copies the data from the address space of the write process to memory. Otherwise, the write process sleeps in the wait queue of the VFS index node, and then the kernel calls the scheduler, and the scheduler chooses other processes to run. The write process is actually in an interruptible wait state, and the write process will receive a signal when there is enough space in the memory to hold the write data, or when the memory is unlocked, the read process wakes the write process. When the data is written to memory, the memory is unlocked, and all the read processes that hibernate on the index node are awakened.

The read process of the pipeline is similar to the write process. However, the process can return an error message immediately when no data or memory is locked, rather than blocking the process, depending on the open mode of the file or pipeline. Conversely, a process can hibernate the wait queue in the index node waiting for the write process to write to the data. When all processes have completed the pipeline operation, the index node of the pipeline is discarded, and the shared data page is freed.

Because the implementation of the pipeline involves a lot of file operations

command to get the size of the Linux memory page (Base page): Getconf page_size, the general output is 4096, which is 4KB.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/84/51/wKioL1eMw_LQ5yg6AAAK_k93Gqc881.png "title=" Getconf_page_size.png "alt=" Wkiol1emw_lq5yg6aaak_k93gqc881.png "/>

Capacity of the pipe

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/84/51/wKiom1eMxAzx35YiAAA7meBQhxY403.png "style=" float: none; "title=" P1.png "alt=" Wkiom1emxazx35yiaaa7mebqhxy403.png "/>

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/84/51/wKioL1eMxAzBa_QqAABM6n2nHs4172.png "style=" float: none; "title=" P2.png "alt=" Wkiol1emxazba_qqaabm6n2nhs4172.png "/>

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/84/51/wKiom1eMxAzSX4WNAABhK-9__6Y531.png "style=" float: none; "title=" P3.png "alt=" Wkiom1emxazsx4wnaabhk-9__6y531.png "/>

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/84/51/wKiom1eMxA2wBjAAAAASZ2BjLt8717.png "style=" float: none; "title=" P4.png "alt=" Wkiom1emxa2wbjaaaaasz2bjlt8717.png "/>

"64KB"

Features of the piping:

1 Dependent file system

2 One-way communication, to two-way, you need to create two pipelines

3 allow a blood-related process to communicate

4 byte stream oriented

5 pipelines Follow the process, the process is in the pipeline, the process disappears, and the pipeline disappears, and two processes disappear.

Four special cases of piping:

The case 1:pipe The write end reads until the remaining data is read, and the last read returns 0, as if the end of the file was read.

Test code:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M00/84/51/wKiom1eMxYeDaW3BAABZDQtGv2E471.png "style=" float: none; "title=" Image 1.png "alt=" Wkiom1emxyedaw3baabzdqtgv2e471.png "/>

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M00/84/51/wKiom1eMxYewuYQPAAA9VHkvnAg248.png "style=" float: none; "title=" Image 2.png "alt=" Wkiom1emxyewuyqpaaa9vhkvnag248.png "/>

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/84/51/wKioL1eMxYihSF9EAAAQLjakg4c708.png "style=" float: none; "title=" Image 3.png "alt=" Wkiol1emxyihsf9eaaaqljakg4c708.png "/>

Scenario 2: The write end is not closed, but the write end does not write the data, read the end of the read, read the data again after reading will block until there is data to re-read.

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/84/51/wKiom1eMxaTQ3Mo1AAAq9Prw44I624.png "title=" Picture 4.png "alt=" Wkiom1emxatq3mo1aaaq9prw44i624.png "/>

situation 3: Read end, write end ,then process will receive signal sigpipe, usually cause the process to terminate abnormally

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/84/51/wKioL1eMxcrCtxPPAAHcuyvxCbw254.png "style=" float: none; "title=" Image 5.png "alt=" Wkiol1emxcrctxppaahcuyvxcbw254.png "/>

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/84/51/wKiom1eMxcqyQJnQAADNx6JvQbg806.png "style=" float: none; "title=" Image 6.png "alt=" Wkiom1emxcqyqjnqaadnx6jvqbg806.png "/>

situation 4: Read end not closed, read end also not read

The write end is written all the time, and when the pipe is full, the write will block until there is an empty position in the pipeline to write.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/84/51/wKioL1eMxfnC6WHFAADHIAFGgm4159.png "style=" float: none; "title=" Image 7.png "alt=" Wkiol1emxfnc6whfaadhiafggm4159.png "/>

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/84/51/wKiom1eMxfrRISL9AAAM6eOhWy0377.png "style=" float: none; "title=" Image 8.png "alt=" Wkiom1emxfrrisl9aaam6eohwy0377.png "/>

Blocking:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M00/84/51/wKiom1eMxfqDstYsAAAq2ArXG0g678.png "style=" float: none; "title=" Image 9.png "alt=" Wkiom1emxfqdstysaaaq2arxg0g678.png "/>

This article is from the "City Hunter" blog, please be sure to keep this source http://alick.blog.51cto.com/10786574/1827499

Process Communication----pipeline (pipe)