Shell redirection and pipelines)

What is data stream redirection?

What is data stream redirection? This should be explained by the running result of the command! Generally, if you want to run a command, it will usually be like this:

Figure 5.1.1 data transmission during command running

When we run a command, the command may read data from the file and output the data to the screen after processing. In this example, standard output and standard error output represent "standard output" and "standard error output" respectively. These two items are output to the screen by default! So what is standard output and standard error output?

 

Standard output and standard error output

To put it simply,Standard output refers to the "correct information returned by the Command run", and the standard error output can be understood as "error information returned after the command fails to run 』. For example, our system has/etc/crontab by default, but no/etc/vbirdsay, at this time, if you issue the "cat/etc/crontab/etc/vbirdsay" command, cat will:

• Standard output: Read/etc/crontab and display the file content on the screen;
• Standard Error output: the error message is displayed on the screen Because/etc/vbirdsay cannot be found.

No matter whether the correct or wrong data is output to the screen by default, the screen is messy! Can we use some mechanisms to separate the two data sets? Of course! That is the function of data stream redirection! Data Stream redirection can transfer standard output (stdout) and standard error output (stderr) to other files or devices respectively, and the special characters used for transmission are as follows:

1. Standard input (stdin ):Code0, use <or <;
2. Standard output (stdout): the code is 1, use> or>;
3. Standard Error output (stderr): Code 2, Use 2> or 2>;

 
1>: Output "correct data" to a specified file or device in a overwriting method;1>: Outputs "correct data" to the specified file or device in an accumulative way;2>: Overwrite the "wrong data" to the specified file or device;2>>: Outputs "wrong data" to the specified file or device in an accumulative way;
 

Note that there are no spaces in between "1>" and "2>!

 

Store stdout and stderr into different files

 
[Dmtsai @ WWW ~] $Find/Home-name. bashrc> list_right2> List_error
 

 

Discard the wrong data and display the correct data on the screen

 
[Dmtsai @ WWW ~] $Find/Home-name. bashrc2>/Dev/Null/Home/dmtsai/. bashrc <= only stdout is displayed on the screen, and stderr is discarded.
 

Write correct and wrong data to the same file

Write all the command data to the file named list [dmtsai @ WWW~] $Find/Home-name. bashrc> List2> List <=Error [dmtsai @ WWW~] $Find/Home-name. bashrc> List2> &1<=Correct [dmtsai @ WWW~] $Find/Home-name. bashrc &> List <= Correct
 

The reason for the first row error in the preceding table is,Because two pieces of data are written to a file at the same time without special syntax, the two pieces of data may be written into the file at the same time, resulting in disordered order.. Therefore, although the final list file will still be generated, the data arrangement in it will be strange, rather than the output sorting on the original screen. The special syntax for writing data to the same file is shown in the table above. You can use 2> & 1 or! Generally, laruence is used to the syntax of 2> & 1!

 

Standard Input: <and <

After learning about stderr and stdout, what is it? Haha! In the simplest statement 『Replace the data originally required by the keyboard with the file content.. Let's start with the cat command below to learn what it is called "keyboard input!

 
ExploitationCatCommand to create a file [root @ WWW~] #Cat>CatfiletestingCat FileTest<= Click [CTRL] +D. Leave [root @ WWW~] #CatCatfiletestingCat FileTest
 

after Cat is added, the catfile is automatically created, and the content is the two rows of data just entered on the keyboard. No! So can I replace the keyboard input with a plain text file? That is to say, Can I replace the keyboard hitting with the content of a file? yes!

  Use stdin to replace keyboard input to create a new file. [root @ WWW  ~] #  CAT  catfile <~ /. bashrc [root @ WWW  ~] # Ll catfile ~ /. bashrc -RW-r --  1  root Root  194  Sep  26   13 :  36 /root/. bashrc -RW-r --  1  root Root  194  Feb  6   18 :  29   catfile # note that the sizes of the two files will be the same! Almost like copying with   CP ! 

This is very helpful! Especially when using commands like mail. After understanding <, It is terrible to come back again <this two consecutive symbols are less. It indicates the ending input character! For example, "I want to use Cat to directly output the input information to the catfile, and when the keyboard inputs EOF, this input ends", then I can do this:

 
[Root @ WWW ~] #Cat> Catfile <"EOF">This is a test.>OK now stop> EOF <= enter this keyword. It ends immediately without entering [CTRL] +D [root @ WWW~] #CatCatfilethis is a test. OK now stop<= Only these two rows do not exist!
 

Have you seen it? With the <control character on the right side, we can terminate an input without entering [crtl] + D! This pairProgramWriting is very helpful! Well, why do we need to use command output for redirection? Let's talk about it!

 

Pipeline command (PIPE)

As mentioned above, some output data will appear when the bash command is run! So how should we configure this group of data to get the desired format after several procedures? This involves pipeline commands (PIPE ),Pipeline commands use the '|' definition symbol! In addition,Pipeline commands are different from continuous commands.! We will explain this further. Let's take an example to illustrate a simple pipeline command.

Suppose we want to know the number of files under/etc/, we can use ls/etc for reference. However, because there are too many files under/etc, the screen will be full in one breath ~ I don't know what the previous output is? At this point, we can use the assistance of the less command:

 
[Root @ WWW ~] #Ls-Al/etc |Less
 

In this way, the content output by using the LS command can be read by less, and with the less function, we can flip the relevant information before and after! Very convenient, right? Let's take a look at the purpose of this pipeline command "|! ActuallyThe pipeline command "|" can only process the correct information sent through the previous command, that is, the standard output information. It does not directly process stdandard errors.. The overall pipeline command can be expressed as follows:

Figure 6.1.1 pipeline command processing

The first data connected to each pipeline must be a "command! AndThis command must be able to accept standard input data.Only in this way can such commands be "pipeline commands". For example, less, more, Head, and tail commands can all accept standard input pipeline commands. For example, ls, CP, and MV are not pipeline commands! Because ls, CP, and MV do not accept data from stdin. That is to say, there are two main points to note about pipeline commands:

• Pipeline commands only process standard output. Standard Error output is ignored.
• Pipeline commands must be able to accept the data from the previous command as standard input for further processing.

  Usage of minus signs

Pipeline commands are very important in Bash's continuous processing program! In addition, Log File analysis is also a very important part, so please pay special attention to it! In addition, in pipeline commands, stdout of the previous command is often used as the stdin. When some commands need to use file names (such as tar) for processing, this stdin and stdout can be replaced by minus signs "-". For example:

 
[Root @ WWW ~] #Tar-CVF-/home |Tar-Xvf-
 

The above example is: "I pack the files in/home, but the packaged data is not recorded in the file, but transmitted to stdout. After going through the pipeline, send tar-CVF-/home to tar-xvf -』. The following-is the stdout that uses the previous command. Therefore, we do not need to use file! This is a common example! Note!

 

From http://vbird.dic.ksu.edu.tw/linux_basic/0320bash_5.php

Http://vbird.dic.ksu.edu.tw/linux_basic/0320bash_6.php#pipe_7