From windows to * nix (5) IO redirection and Pipelines

Standard IO

 

Stdin, stdout, and stderr are standard inputs, standard outputs, and standard errors. Windows and * nix.

By default, the standard input is equivalent to the keyboard input, and the standard output and standard error are equivalent to the screen output.

In C language, these three are FILE * types (FILE pointers). For example, if you use printf, the content is directly output to the standard output, which is equivalent to fprintf (stdout ,...); you use scanf to read data directly from standard input. It is equivalent to fscanf (stdin ,...).

 

What are their functions? For windows, it seems that it can only be used to write experimental small programs.

 

In fact, * nix is an important part of the user interface and can also be used for redirection.

 

 

IO redirection

 

The basic usage of redirection is as follows in * nix shell:

~ /Test $ echo Hello <br/> ~ /Test $ echo Hello> a.txt <br/> ~ /Test $ cat a.txt <br/> Hello <br/> ~ /Test $ 

Here, the echo command is used to output as is,> redirects the standard output to the file, and then uses the cat command to display the content of the file.

Let's look at another example of the redirection standard input:

~ /Test $ wc <a.txt <br/> 1 1 6 <br/> ~ /Test $ 

Here, the <number is used to send the file content to the wc program to count the number of characters.

 

MPs queue

 

The input and output redirection in the preceding shell is only limited to files, while Pipelines provide a common method.

 

~ /Test $ cat a.txt | wc <br/> 1 1 6 <br/> ~ /Test $ 

 

Pipeline character | connects the output and input of the two processes. The number of statistical characters is also achieved.

 

Similar usage is also the same in windows cmd. If you are interested, you can explore it.

However, a major difference between the two is that * nix has a software model called filter, which is not available in windows software.

 

Filter

 

Because of the existence of the filter mode, * nix can use a large set of gadgets to achieve "One Line of shell is better than ten thousand lines of C ".

 

A filter is a program action that reads one or more files specified by the command line in sequence. If no file name is specified, the filters are read from the standard input, then write the processing result to the standard output. If any error occurs, write the error to the standard error.

 

From the above description, we can easily imagine that multiple programs that follow the filter mode can be chained with pipeline characters. Information is transmitted through standard input and standard output, and error messages are sent to standard errors to avoid interfering with the real information flow.

 

 

General purpose filters that are frequently used in daily use include the following:

 

Cat copies input to output

Number of starting lines intercepted by head

Number of trailing lines intercepted by tail

Cut Column

 

Grep performs the row-based search function

Wc executes the counting function

Sort by sort

 

Some instances that use filters (you can use these examples in segments to understand the meaning of each part ):

Ps-ef | grep httpd | grep-v grep | wc-l

Number of httpd processes in the current system

 

Cut-d ''-f 1 access. log | sort | uniq-c | sort-nr

Extract A column from the apache Log File and sort it by the number of occurrences.

 

 

Even if the filter mode is not strictly followed, as long as the program has a standard output, it can still be used as the starting position of the pipeline. In addition, it can be easily reused by other programs.

 

For example, to obtain the file list in windows, you have to use the API provided by the operating system or the C library function.

However, in * nix, you can directly call the ls command to obtain the result.

$ Ls-FLa> ~ /Dir. log <br/> $ cat ~ /Dir. log <br/>. /<br/> .. /<br/> bin/<br/> boot/<br/> dev/<br/> etc/<br/> home/<br/> lib/<br/> lost + found/<br/> mnt/<br/> opt/<br/> proc/<br/> root/<br/> sbin/<br/> sys /<br/> tmp/<br/> tony. tgz <br/> usr/<br/> var/<br/> $ 

As a counterexample, let's look at how the dir command in windows is designed.

 

C:/> dir <br/> the volume in drive C has no labels. <Br/> the serial number of the volume is the B0B5-18AD <br/> C:/directory <br/> <DIR> 360Rec <br/> 0 AUTOEXEC. BAT <br/> 0 CONFIG. SYS <br/> <DIR> dell <br/> <DIR> Documents and Settings <br/> <DIR> Downloads <br/> <DIR> i386 <br/> 4,128 INFCACHE.1 <br/> <DIR> Program Files <br/> <DIR> Temp <br/> <DIR>> WINDOWS <br/> 3 files, 4,128 bytes <br/> 8 directories, 9,779,306,496 available bytes <br/> C: /> 

It is very difficult to extract a file list from this result, which is much more difficult than using APIs or library functions. (Dir/w looks better, but it has more problems)

It should be said that at the beginning of the design, the windows command line tool did not want to interact with other programs.

 

 

 

Application of pipelines in programs

 

As we can see above, the input and output can be redirected not only to files, but also to other processes through pipelines. With this feature, we can redirect it to the network, if you have written CGI in C, you can understand this. Imagine that we can implement a network protocol server simply by using standard IO functions such as scanf and printf without socket, the actual establishment of network connections is achieved through a general network service program. For example, tcpd establishes a common tcp-based network service. Whenever a connection comes in, tcpd starts your program and receives data that will be written into your standard input, and your standard output will be sent to the client by tcpd.

 

With similar development ideas, we can add SSL support for your server programs without modifying any code.

 

Pipelines are also implemented in windows, but applications are not very common due to the Development atmosphere or other technical reasons.

 

For pipeline programming, the general process of windows and * nix is the same, windows: 1. create an MPS queue for CreatePipe (& hReadPipe, & hWritePipe ...) 2. the STARTUPINFO struct required for the function to create a new process CreateProcess. It has three members: hStdOutput, hStdInput, and hStdErr. You can pass in the preceding hReadPipe or hWritePipe as needed. If necessary, you can create many-to-many pipelines in step 1. If you want to write data, you can read the data to the sub-process. If you want to read the data, you can write the data to the sub-process. 3. Write Data to the standard input of the new process, or read data from the standard output of the new process. Before reading or writing, use CloseHandle to turn off the handle (one of hReadPipe and hWritePipe) of the passed sub-process. Then read and write with ReadFile or WriteFile. * Nix: 1. create a pipeline for pipe (int pipefd [2]) 2. create a process first fork, then the sub-process uses dup2 to copy pipefd [0] To STDIN_FILENO (that is, 0), and then disable pipefd [1]. or copy pipefd [1] to STDOUT_FILENO (that is, 1), and disable pipefd [0]. Call exec to load the process to be created. 3. read or write is used for reading or writing. Before reading and writing data, close the sub-process using the close function. The difference is that * nix creates a process in two steps: fork and exec. Windows is only one step. Difference 2: * nix needs to manually dup to copy the file descriptor. windows has done this inside the API for creating the process. * Nix has many steps, but is flexible. It can change multiple modes. For example, after fork, not only the child process can execute, but also the parent process can execute, completely implement functions such as "ls | wc-l. That is, the child process is replaced by wc, and the parent process is replaced by ls. Windows has few steps and the mode is fixed. To implement the above functions, it is more complicated. You need to create two processes, connect them, and wait for the two processes to end.