MPs queue details

Linux Pipelines

Pipeline is an important communication method in Linux. It connects the output of a program directly to the input of another program. It is often said that most pipelines refer to unknown pipelines, an unknown pipeline can only be used between unrelated processes, which is the biggest difference between it and a famous Pipeline.

A famous pipeline is named pipe or FIFO (first-in-first-out). It can be created using the mkfifo () function.

Implementation Mechanism of Linux Pipelines

In Linux
Pipeline is a frequently used communication mechanism. In essence, an MPS queue is also a file, but it is different from a common file. The MPs queue can overcome the following two problems:

·


Limit the MPs queue size. In fact, a media transcoding queue is a fixed buffer. In Linux
The buffer size is 1
Page, that is, 4 K
The size of a byte.
Files grow without testing. Using a single fixed buffer can also cause problems. For example, the pipeline may be full when writing, and when this happens, the pipeline's write ()
The call will be blocked by default, waiting for some data to be read to free up enough space for write ()
Call write.

·


Reading a process may also work better than writing a process.
Fast. When data of all current processes has been read, the MPs queue becomes empty. When this happens, a subsequent read ()
The call will be blocked by default, waiting for some data to be written, which solves the read ()
The end of the file returned by the call.

Note: Reading data from an MPS queue is a one-time operation. Once the data is read, it is discarded from the MPs queue and space is released to write more data.

1. Pipeline Structure

In Linux
Pipeline does not use a dedicated data structure, but uses the file system's file
Structure and VFS
Inode
. By
Structure pointing to the same temporary VFS
Index node, and this VFS
The index node points to another physical page.

2. MPS queue read/write

The source code of pipeline implementation is in FS/pipe. c
In pipe. c
There are many functions, two of which are more important, namely, the pipeline READ function.
Pipe_read ()
And pipeline write function pipe_wrtie ()
. The pipeline write function copies bytes to VFS
The index node points to the physical memory and writes data, while the pipeline reads the Function
The data is read by copying the bytes in the physical memory. Of course, the kernel must use a certain mechanism to synchronize access to the pipeline. Therefore, the kernel uses locks, waiting queues, and signals.

When the write process

When writing to an MPS queue, it uses the standard library function write ()
The system can find the file of the file according to the file descriptor passed by the library function.
Structure. File
The address of the write function is specified in the structure. Therefore, the kernel calls this function to complete the write operation. Before writing data to the memory, the write function must first check the VFS
The actual memory replication can be performed only when the information in the index node meets the following conditions:

 

·
The memory has enough space to accommodate all the data to be written;

·
The memory is not locked by the read program.

 

If both of the preceding conditions are met, the write function first locks the memory and then
To the memory. Otherwise, the write process will sleep in VFS
Suo
In the waiting queue of the cited node, the kernel will call the scheduler, And the scheduler will select other processes to run. The write process is in an interrupted wait state. When the memory has enough space to accommodate the write
When data or memory is unlocked, the read process will wake up the write process, and the write process will receive a signal. When data is written to the memory, the memory is unlocked, and all reading processes that sleep on the index node are called
Wake up.

Management
The reading process is similar to the writing process. However, a process can return an error message immediately when there is no data or the memory is locked, rather than blocking the process, depending on the open mode of the file or pipeline. Otherwise, the process can
Wait for the write process to write data in the waiting queue of the index node to sleep. After all the processes complete the pipeline operation, the indexing node of the pipeline is discarded, and the shared data page is also released.

Because the implementation of pipelines involves many file operations,
Therefore,
When the reader finishes learning about the file system, he reads pipe. C.
You will find it difficult to understand the code in.

The creation and use of Linux pipelines are simpler. The only reason is that it requires fewer parameters. To create the same MPs queue as windows, use the following code snippet for Linux and UNIX:

Create a Linux Named Pipe
Int fd1 [2];

If (pipe (fd1 ))

{

Printf ("pipe () failed: errno = % d", errno );

Return 1;

}

The Linux pipeline has a limit on the size of the previous write operation to block. The kernel-level buffer specifically used for each pipeline is exactly 4096 bytes. Unless the reader clears the MPs queue, a write operation exceeding 4 kb will be blocked. In fact, this is not a limit, because read and write operations are implemented in different threads.

Linux also supports named pipelines. Early commentators suggested that I compare the Linux naming pipeline with the Windows naming pipeline for fairness. I wrote another program that uses Named Pipes in Linux. I found that for the named and unnamed pipelines in Linux, the results are no different.

Linux pipes are much faster than Windows 2000 named pipes, while Windows 2000 named pipes are much faster than Windows XP Named Pipes.

 

Example:

# Include <stdio. h>
# Include <unistd. h>

Int main ()
{
Int N, FD [2]; // The FD here is an array of file descriptors for creating pipelines.
Pid_t PID;
Char line [100];
If (pipe (FD) <0) // create an MPS queue
Printf ("pipe create error/N ");

If (pid = fork () <0) // use fork () to create a new process
Printf ("fork error/N ");

Else if (pid> 0) {// here is the parent process. First close the read end of the pipeline, and then write "Hello World" to the write end of the pipeline"
Close (FD [0]);
Write (FD [1], "Hello word/N", 11 );
}
Else {
Close (FD [1]); // This is a sub-process. First, close the write end of the pipeline, and then read the data from the read end of the pipeline.
N = read (FD [0], line, 100 );
Write (stdout_fileno, line, N );
}
Exit (0 );
}