GNU/Linux Application Programming: Using pipelines for programming

Introduction

In this article, we will learn about the GNU/Linux pipeline. Although the pipeline model is very old, it is still a very useful inter-process communication mechanism. We will learn what a semi-bidirectional pipeline is and what a famous pipeline is. They all provide a FIFO (first-in-first-out) queuing model to allow inter-process communication.

MPs Queue Model

A Visualized pipeline is described as a unidirectional connector between two entities. For example, let's take a look at the following GNU/Linux Command:

Ls-1 | WC-l

This command creates two processes, one associated with LS-l and the other associated with WC-L. Then it connects the two processes (11.1) by setting the standard input of the second process to the standard output of the first process ). The result is that the number of files in the current subdirectory is calculated.

Our command sets a pipeline between two GNU/Linux commands. After the command LS is executed, its output is used as the second command WC.
(Word
Count. This is a one-way pipeline-Communication occurs in one direction. The connection between the two commands is completed by GNU/Linux. We can also do this in applications.
(We will prove this later ).

Anonymous pipeline and named Pipeline

An anonymous pipeline or one-way pipeline provides a method for a process to communicate with a sub-process (anonymous type. This is because no operating system is available
To find an anonymous process. Its most common usage is to create an anonymous pipeline in the parent process, and then pass the pipeline to its child process, and then they can communicate. Note:
For two-way communication, we should consider using the socket (sockets) API.

Another type of MPs queue is the famous MPs queue. The function of a famous pipeline is similar to that of an anonymous pipeline. The difference is that it can exist in a file system and can be found by all processes. This means that processes without kinship can use it for communication.

In the following sections, we will learn famous and anonymous pipelines at the same time. We will make a quick tour of the pipeline, and then we will learn in detail the pipeline API and the GNU/Linux system-level commands that support pipeline programming.

Whirlwind tour

Let's start our whirlwind tour with a simple pipeline programming model example. In this simple example, we create a pipeline in a process and write a message to it. The next step is to read the previously written message from this pipeline, then display it.

Listing 11.1: A simple pipeline example

1:#include 2:#include 3:#include 4: 5:#define MAX_LINE 80 6:#define PIPE_STDIN 0 7: #define PIPE_STDOUT 1 8: 9: int main() 10: { 11: const char *string={"A sample message."}; 12: int ret, myPipe[2]; 13: char buffer[MAX_LINE+1]; 14: 15: /* Create the pipe */ 16: ret = pipe( myPipe ); 17: 18: if (ret == 0) { 19: 20: /* Write the message into the pipe */ 21: write( myPipe[PIPE_STDOUT], string, strlen(string) ); 22: 23: /* Read the message from the pipe */ 24: ret = read( myPipe[PIPE_STDIN], buffer, MAX_LINE ); 25: 26: /* Null terminate the string */ 27: buffer[ ret ] = 0; 28: 29: printf("%s/n", buffer); 30: 31: } 32: 33: return 0; 34: }

In listing 11.1, we used the pipe function in line 16 to create our pipeline. We passed in a pipe function with two elements representing our
An integer array of pipelines. The pipeline is defined as a pair of separate file descriptors-one input and one output. We can write data from one end of the pipeline and read data from the other end of the pipeline. If the MPs queue is successfully created,
The API function pipe returns 0. Based on the returned results, mypipe will include two new input files (mypipe [1]) representing the pipeline and output files (mypipe [0]) representing the pipeline.
File descriptor. (PS:
The in file descriptor indicates that the file or device associated with it is equivalent to an input device; The out file descriptor indicates that the file or device associated with it is equivalent to an output device .)

In line 21, we use the function write to write our messages to the pipeline. We specify the stdout (standard output) Descriptor (this is from the application's
From the perspective, rather than the pipeline ). The pipeline now contains our messages, which can be read by the function read in 24 rows. Here we will explain again that we have used
Stdin (standard input) descriptor to read from the pipeline. The function read saves the messages read from the pipeline to the variable buffer. To make the buffer truly a character
String, we add a 0 (null) after it, and then we can use the printf function to display it in line 29.

Although this example is interesting, we can use any number of mechanisms to execute communication between ourselves. Next we will learn more complex examples of communication between processes (whether related or unrelated.

Detailed Review

Although the vast majority of pipeline models are function pipe, there are two other functions that should be discussed on their availability Based on pipeline programming. Table 11.1 lists the functions that we will discuss in detail in this chapter:

API function usage

Pipe creates an anonymous Pipeline

DUP creates a copy of a file descriptor

Mkfifo creates a famous pipe (first-in-first-out)

At the same time, we will also learn some other functions that can be used for pipeline communication, especially those that can use pipeline for communication.

Note: We must remember that a pipeline is just a file descriptor, so any function that can operate on the file descriptor can use the pipeline. This includes select, read, write, fcntl, and freopen, but not just these functions.

Pipe

The API function pipe creates a new pipeline represented by an array containing two file descriptors. The pipe function is prototype as follows:

# Include

Int pipe (int fds [2]);

If the pipe function is successful, it returns 0; otherwise, it returns-1, and errno is set as appropriate. If the returned result is successful, the array FDS
(It is passed by reference) will have two active file descriptors. The first element of the array is a file descriptor that can be read by the application. The second element is a file that can be written by the application.
Descriptor.

Now let's look at a more complex example of the next application pipeline in a multi-process application. In this application, we will create a pipeline in line 14,
Then, use the fork function in line 16 to create a new process (child process) in the parent process of the program ). In the sub-process, we try to read from the input file descriptor of our pipeline (18
But if there is nothing to read, we will suspend this sub-process. After the read operation, we terminate the string through a null (that is, to make an array character a word
String), and then print the string we read. The parent process simply writes a test string to the pipeline by using the output (write) file descriptor (the second element of the array in the pipeline structure), and then uses
The function wait waits for the sub-process to exit.

Note: apart from the fact that our child processes inherit the file descriptors created by the parent process using the pipe function, they are used to communicate with another child process or parent process.
Letter, there is no more eye-catching about this program. Recall: Once the fork function is completed, our processes are independent (except that child processes inherit the features of the parent process, such as pipeline descriptions
Description ). Since the memory is separated, the pipe method provides an interesting model for inter-process communication.

Listing 11.2 illustrates the pipeline model between two processes.

1: #include 2: #include 3: #include 4: #include 5: 6: #define MAX_LINE 80 7: 8: int main() 9: { 10: int thePipe[2], ret; 11: char buf[MAX_LINE+1]; 12: const char *testbuf={"a test string."}; 13: 14: if ( pipe( thePipe ) == 0 ) { 15: 16: if (fork() == 0) { 17: 18: ret = read( thePipe[0], buf, MAX_LINE ); 19: buf[ret] = 0; 20: printf( "Child read %s/n", buf ); 21: 22: } else { 23: 24: ret = write( thePipe[1], testbuf, strlen(testbuf) ); 25: ret = wait( NULL ); 26: 27: } 28: 29: } 30: 31: return 0; 32: }

Note: In these simple programs, we did not discuss how to close the pipeline because the resources associated with the pipeline will be automatically released once a process ends. Even so, calling the function close to close the descriptor of the pipeline is a very good programming practice, such:

Ret = pipe (mypipe );

...

Close (mypipe [0]);

Close (mypipe [1]);

If the write end of the MPs queue is closed and a process wants to read data from the MPs queue, a 0 value is returned. This means that the MPs queue is no longer used and should be shut down.
. If the read end of the MPs queue is closed and a process wants to write data to the MPs queue, a signal is generated. This signal (which will be discussed in chapter 12th-Introduction to socket programming) is called
Sigpipe. Applications that want to write data to pipelines usually contain a signal handle that captures such a situation.

DUP and dup2

The DUP and dup2 functions are very useful for copying a file descriptor. They are often used to redirect stdin, stdout, or stderr of a process. The DUP and dup2 functions are prototype as follows:

# Include

Int DUP (INT oldfd );

Int dup2 (INT oldfd, int targetfd );

The DUP function allows us to copy a descriptor. We pass an existing descriptor to the function, and then it returns a new description identical to the previous one.
. This means that the two descriptors share the same structure. For example, we use a file descriptor to execute an lseek (search in the file). The current position of the file (internal pointer of the file) is in the second
The same is true. The use of the function DUP is shown in the following code snippet:

Int fd1, fd2;

...

Fd2 = DUP (fd1 );

Note: creating a descriptor before fork calls produces the same effect as calling the function DUP. The sub-process receives a replication descriptor, just as it imitates the DUP call.

The function dup2 is similar to dup, but it allows the caller to specify the ID of an active descriptor that is already the target descriptor. After the function dup2 returns successfully, the new target descriptor copies the first Descriptor (targetfd = oldfd ). Let's take a look at the following code snippet to illustrate dup2 usage:

Int oldfd;

Oldfd = open ("app_log", (o_rdwr | o_create), 0644 );

Dup2 (oldfd, 1 );

Close (oldfd );

In this example, we open a new file named "app_log" and receive a file descriptor named fd1. We use
Oldfd and 1 are called.
Dup2, So we replaced file descriptor 1 (stdout standard output) with oldfd (our new file ). Any data written to stdout of standard output will be written now
In a file called "app_log. Note that we disable the oldfd after copying it. However, this does not close the newly opened file because file descriptor 1 is
It -- we can think so.

Now let's learn a more complex example. Let's look back at the previous section in this chapter to make the LS-L output a WC-l input. We now use a c Application to explore how to allow this example (listing 11.3 ).

At the beginning of listing 11.3, we created our pipeline in nine rows and then created the child process of the program in row 13-16. Then we created the parent process in Row 20-23.
Process. Stdout (standard output) descriptor is disabled in row 13. The sub-process provides the LS
-L is not written to stdout. On the contrary, it is written to our pipeline (DUP is used for redirection ). In line 14, we used dup2 to redirect stdout to us.
Pipeline (PFDS [1. Once such an operation is completed, we close the input of our pipeline (because it will never be used ). Finally, we use the Function
Execlp to replace the image of our sub-process with the image of the command LS-L. Once this command is executed, any output will be transmitted to the input.

Now let's take a look at the receiver of the pipeline. The parent process acts as such a role and follows a similar pattern. We first shut down the service in 20 rows.
Stdin (standard input), because we will not receive any data from it. Next, we will use the function dup2 (line 21) to make stdin the output end of the pipeline. This is
0 (usually
Stdin) functions become the same as PFDS [0. We disabled the stdout standard output end of the MPs Queue (PFDS [1]) because it is not used here.
(22 rows ). Finally, execlp is used to run the WC-1 (23 rows) command that treats the content in the pipeline as its input ).

Listing 11.3: assembly line command C program

1: #include 2: #include 3: #include 4: 5: int main() 6: { 7: int pfds[2]; 8: 9: if ( pipe(pfds) == 0 ) { 10: 11: if ( fork() == 0 ) { 12: 13: close(1); 14: dup2( pfds[1], 1 ); 15: close( pfds[0] ); 16: execlp( "ls", "ls", "-1", NULL ); 17: 18: } else { 19: 20: close(0); 21: dup2( pfds[0], 0 ); 22: close( pfds[1] ); 23: execlp( "wc", "wc", "-l", NULL ); 24: 25: } 26: 27: } 28: 29: return 0; 30: }

In this program, 01:10 is important and worth writing down-this is how our sub-process redirects its own output to the input of the pipeline, the parent process redirects its own input to the output of the pipeline-a very useful technique worth remembering.

Mkfifo

The mkfifo function is used to create a file (also known as a pipe) that provides the FIFO function in the file system ). Currently
All we are talking about is the anonymous pipeline. They are dedicated to communication between a parent process and its Child processes. Famous pipelines are visible in the file system and can be used by any process. Function mkfifo
The prototype is as follows:

# Include

# Include

Int mkfifo (const char * pathname, mode_t mode );

The command mkfifo requires two parameters. The first (pathname) is a special file to be created in the file system. The second (mode) specifies the FIFO
Read and write permissions. If the command succeeds, mkfifo will return 0, and-1 will be returned if the command fails (and errno will be written as appropriate ). Let's take a look at the next creation using the mkfifo Function
The following is an example of FIFO:

int ret; ... ret = mkfifo( "/tmp/cmd_pipe", S_IFIFO | 0666 ); if (ret == 0) { // Named pipe successfully created } else { // Failed to create named pipe }

In this example, we use the pipeline _pipe file to create a FIFO (famous Pipeline) in the sub-directory/tmp ). Next, we open this file for read and write.
It can be used for communication. Once we open a famous Pipeline, we can use a typical I/O command to read and write. For example, the following is a generation that uses fgets to read data from the pipeline.
Code segment:

PFP = fopen ("/tmp/pai_pipe", "R ");

...

Ret = fgets (buffer, max_line, PFP );

We can use the following code segment to write the pipeline for the above Code:

PFP = fopen ("/tmp/pai_pipe", "W + );

...

Ret = fprintf (PFP, "here's a test string! /N ");

The interesting thing about famous pipelines is that they work in a place that is seen as a collection point. We will discuss this in the discussion of the system command mkfifo.
. A reader cannot open a famous Pipeline unless a writer actively opens the other end of the pipeline. The reader will be blocked when calling the OPEN function until a writer appears. Although
With such a restriction, the famous pipeline is still a useful mechanism for inter-process communication.

System commands

Let's take a look at a system command related to the MPs pipeline model. The command mkfifo is the same as the API function mkfifo, allowing us to create a famous pipeline on the command line.

Mkfifo

Command mkfifo is one of the two methods for creating a famous pipe (Special FIFO file) on the command line. The command mkfifo is generally used as follows:

Mkfifo [Options] Name

Here, options-M indicates the mode (permission), and name indicates the name of the famous pipeline to be created (including the path if needed ). If no permission is specified, the default value is 0644. The following is an example. It creates a famous Pipeline named pipeline _pipe in the directory/tmp:

$ Mkfifo/tmp/cmd_pipe

We can simply adjust the option by using option-M. The following is an example of setting the permission to 0644 (but we must first Delete the original ONE ):

$ RM pai_pipe

$ Mkfifo-M 0644/tmp/pai_pipe

Once the permission is created, we can use this pipeline to communicate on the command line. On another terminal, we use the echo command to write to the famous Pipeline pai_pipe:

$ Echo HI> pai_pipe

When this command ends, our reader will be awakened and ended (the sequence of the reader commands is clearly completed below ):

$ Cat pai_pipe

Hi

$

The preceding example shows that a famous pipeline can be used not only in C programs but also in scripts.

You can also use the mknod command to create a famous Pipeline (the special file that follows it can be many other types of files ). Run the mknod command to create a famous Pipeline;

$ Mknod pai_pipe P

Here, the famous Pipeline pipeline _ pipe is created in the current subdirectory (P is the type of the famous pipeline ).

Summary

In this chapter, we made a whirlwind tour of famous and anonymous pipelines. We reviewed the program and command line methods for creating pipelines, and also reviewed
Typical communication methods
I/O mechanism. We also reviewed how to use the DUP and dup2 functions for I/O redirection. Although pipelines are very useful, these commands or functions are still very useful in a specific scenario.
(Whether a file descriptor is used in the above area, such as a socket or file ).

Pipeline programming API

# Include

Int pipe (INT filedes [2]);

Int DUP (INT oldfd );

Int dup2 (INT oldfd, int targetfd );

Int mkfifo (const char * pathname, mode_t mode );