Deep understanding of pipe

Reprint: http://blog.ddup.us/?p=285

There are several ways to communicate interprocess communication in Linux: Pipe, FIFO, shared memory, semaphores, message queues, shared files, and so on. Pipe and FIFO are the most widely used, the difference between the two is pipe as anonymous pipe, can only be used in a parent-child relationship between interprocess communication, and FIFO can be obtained through a file system, so not subject to the above restrictions. As a channel for communication between parent and child processes, pipe can also be considered as a first-in, first-out queue. basic usage of pipe

The use of pipe is simple and the prototype is as follows:

#include <unistd.h>
int pipe (int pipefd[2]);

Typical use is as follows:

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
int main ()
{
    int pfd[2];
    int ret = pipe (PFD);
    if (ret!= 0) {
        perror ("pipe");
        Exit ( -1);
    }
    pid_t pid = fork ();
    if (PID = = 0) {Close
        (pfd[1]);
        Char rdbuf[1024];
        Read (Pfd[0], rdbuf, sizeof (RDBUF));
        return 0;
    } else if (PID > 0) {close
        (pfd[0]);
        Char wrbuf[] = "ABCD";
        Write (pfd[1], wrbuf, sizeof (WRBUF));
        pid_t cpid = Waitpid ( -1, NULL, 0);
    } else {
        perror ("fork");
    }
    return 0;
}

As can be seen from the code, after invoking pipe, PFD is a pair of associated pipelines, where pfd[0] corresponds to the data input, pfd[1 the corresponding data output. When the parent process wants to send data to the child process, write the data directly into pfd[1], and the corresponding subprocess is read from pfd[0. Pipe buffer Limit

As you can see from the parent-child process above, this is the typical 1:1 producer consumer model in the operating system. The pipeline as a link between producer and consumer buffers also involves two problems: 1 the size of the buffer. 2 The problem of mutual exclusion in buffer reading and writing.

By removing the write line of the parent process from the above code, you can test that the child process read Pipe will block if the pipe is empty (the default is blocked, or it can be set to non-blocking, so that it returns 1 and errno is set to Eagain).

To test the size of the buffer, modify the code again so that the child process no longer reads the data, and the parent process writes the data to it, as follows:

pid_t pid = fork ();
    if (PID = = 0) {Close
        (pfd[1]);
        Pause ();
        return 0;
    } else if (PID > 0) {close
        (pfd[0]);
        Pause ();
        Char buf[] = "a";
        for (int i = 0;; i++) {
            write (pfd[1], buf, sizeof (buf[0));
            printf ("Write pipe%d, total%d\n", i+1, (i+1) *sizeof (buf[0));
        pid_t cpid = Waitpid ( -1, NULL, 0);
    } else {
        perror ("fork");
    }

The modified parent process writes one character to the pipe each time and prints the number of bytes already written. When the program stops, the results displayed on the Linux2.6.27 are:

Write pipe 65536, total 65536

As you can see, the buffer size of the pipe is now 64KB. The same program appears in MACOSX as:

Write pipe 16384, total 16384

You can see that the pipe default buffer size for MacOSX is 16KB.

Given the size of the pipe default buffer, it is natural to think that the buffer size can be manually set. After a search, I found that a lot of people said that this value is limited to dead, in the kernel code fixed is 64K. Later, a copy of the Linux 3.0 code found that the default value can be changed. Related code in INCLUDE/LINUX/PIPE_FS_I.H:

#define Pipe_def_buffers    ...
... /* for F_SETPIPE_SZ and F_GETPIPE_SZ
/long pipe_fcntl (struct file *, unsigned int, unsigned long arg);

See Pipe_def_buffers is the default pipe buffer size, but in pages, the default page size is 4K, so the default pipe buffer size is 64KB. But at the bottom there is a function pipe_fcntl, with two constants F_SETPIPE_SZ, F_GETPIPE_SZ. It should be used to modify the default buffer size.

Sure enough, this feature was added to the 2.6.35 kernel. The release notes can be seen here, and a corresponding commit is seen in the corresponding commit log. This allows you to change the pipe buffer size by using FCNTL with the top two constant directives. The maximum value of the pipe buffer can be easily viewed in/proc/sys/fs/pipe-max-size.

It is now known that the Linux default pipe buffer is 64KB, and the kernel after 2.6.35 can change the buffer size using FCNTL. There is another problem, the problem of reading and writing atoms. atomic access to pipe

Also in include/linux/pipe_fs_i.h, the following code is available:

* Differs
   from pipe_buf to Pipe_size is the length of the actual memory allocation, whereas PIPE_BUF makes Atomi City guarantees.  * *
#define PIPE_SIZE       page_size

See the definition of pipe_size this constant, the size of a page size (default is 4K). You can see from the note that this pipe_size value is the maximum guaranteed number of atomic operations bytes pipe. Also written in the Write man page:

Write requests of {Pipe_buf} bytes or less shall not is interleaved with data to other processes doing writes on the SAM E pipe. 
Writes of greater than {Pipe_buf} bytes may have data interleaved, on arbitrary boundaries, with writes by other processes , whether or not the O_nonblock flag of the "file status" is set.

This pipe_buf constant is defined in/usr/include/linux/limits.h:

#define PIPE_BUF        4096/    * bytes in atomic write to a PIPE * *

You can also view this restriction by ulimit-p, but this is a read-only value and cannot be modified. The ulimit-a output is as follows, where the pipe size is:

Core file size          (blocks,-c) 0
data seg size           (Kbytes,-D) unlimited
scheduling Priority (-             e) 0
File size               (blocks,-f) Unlimited
pending signals (-I                 ) 81920
max locked Memory       (Kbytes, L) 32
Max Memory Size         (Kbytes,-m) unlimited
open files                      (-N) 1024
pipe size            (bytes,-p) 8 POSIX message
queues< c18/> (bytes,-q) 819200
real-time Priority              (-R) 0
stack size              (Kbytes,-s) 10240
CPU Time               (seconds,-T) unlimited
max User Processes              (-u) 1024
virtual memory          (Kbytes,-V) unlimited< C29/>file Locks                      (-X) Unlimited

Reference

Http://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html
http://stackoverflow.com/questions/4624071/pipe-buffer-size-is-4k-or-64k
Http://home.gna.org/pysfst/tests/pipe-limit.html
Http://unix.stackexchange.com/questions/11946/how-big-is-the-pipe-buffer

Http://stackoverflow.com/questions/4739348/is-it-possible-to-change-the-size-of-a-named-pipe-on-linux