Redirection and pipelines in Linux

1 redirection

1.1 redirection symbols

> The output is redirected to a file or device to overwrite the original file.
>! The output is redirected to a file or device to forcibly overwrite the original file.
> The output is redirected to a file or device to append the original file.
<Redirect input to a program

1.2 standard error redirection symbols

2> redirect a standard error output to a file or device to overwrite the original file B-shell
2> redirect a standard error output to a file or device and append it to the original file.
2> & 1 redirect a standard error output to the standard output. Note: 1 may represent the standard output.
> & Redirect a standard error output to a file or device to overwrite the original file C-shell
| & Deliver a standard error pipeline to another command as input

1.3 command redirection example

During bash command execution, there are three output statuses:
1. Standard input; Code: 0; or stdin; usage: <
2. standard output: the code is 1; or stdout; the method used is 1>
3. Error output: the code is 2; or stderr; the method used is 2>

[Test @ test] # ls-al> list.txt
Output the displayed result to the list.txt file. If this file exists, it will be replaced!

[Test @ test] # ls-al> list.txt
Add the displayed result to the list.txt file. The file is accumulative and the old data is retained!

[Test @ test] # ls-al 1> list.txt
2> list. err
Correctly output the displayed data to list.txt
The error data is output to list. err.

[Test @ test] # ls-al 1> list.txt 2> & 1
Output the displayed data, whether correct or incorrect, to list.txt! If an error occurs and the correct file is output to the same file, the preceding method is required! Cannot be written in other formats!

[Test @ test] # ls-al 1> list.txt 2>/dev/null
Discard the displayed data and correctly output the data to the list.txt error! /Dev/null, which can be said to be a black hole device. It is null, that is, it is not saved.

1.4 why use command output redirection

• When the screen output information is important and we need to save it;
• When the program in the background execution does not want him to interfere with the normal output results of the screen;
• The execution results of some routine system commands (such as files written in/etc/crontab;
• We already know some possible error messages when executing commands, so we want to discard them with "2>/dev/null;
• When error messages and correct messages need to be output separately.

2 Pipeline command (PIPE)

As mentioned above, some output data may appear during bash command execution. If this data set must go through several procedures before obtaining the desired format, how should I set it? This involves pipeline commands (pipe). Pipeline commands use '| 』.

Example: simple pipeline commands
Suppose we want to read the "Number of times" of the root login from the last command. What should we do?
The steps are as follows:
1. Execute last to retrieve all the login data for this month;
2. Use grep to retrieve the root of the above output data (stdout;
3. Use the wc command to calculate the number of rows in the previous step!
Because the last output is a line that represents one login, as long as you calculate a few lines, it means several logins. In the preceding three steps
Gradually filter data to obtain our data! The entire command can be written as follows: [test @ test bin] # last | grep root | wc
-L
This pipeline command "|" can only process the correct information sent through the previous command, that is, standard output (stdout) information. It has no direct processing capability for stdandard error.

2.1 introduction to basic pipeline commands

• Cut
Syntax: [root @ test/root] # Cut-d "delimiter" [-CF] fields
Parameter description:
-D: The delimiter followed by a space character 』
-C: followed by the "number of characters 』
-F: What blocks are followed?
Example: [root @ test/root] # Cat/etc/passwd | cut-d ":"-F 1
In the passwd file, use the Separator in each line to list the first block! That's the name!
[Root @ test/root] # Last | cut-c1-20
Extract the data from the last row with 1-20 characters!
• Sort
Syntax: [root @ test/root] # Sort [-T separator] [(+ start) (-end)] [-NRU]
Parameter description:
-T separator: Use a separator to separate different intervals. The default Delimiter is tab.
+ Start-end: sorts data from the start interval to the end interval.
-N: Use "numbers only" for sorting (otherwise, it will be sorted by text type)
-R: reverse sorting
-U: the same row that appears, only listed once!
Example: [root @ test/root] # Cat/etc/passwd | sort sorts the listed personal accounts!
[Root @ test/root] # cat/etc/passwd | sort-t: + 2n sort personal accounts by user IDs (separated, the third is ID, but the first code is 0)
[Root @ test/root] # cat/etc/passwd | sort-t: + 2nr reverse order!
• Wc
Syntax: [root @ test/root] # wc [-lmw]
Parameter description:
-L: number of rows
-M: The number of characters.
-W: How many words
Example: [root @ test/root] # cat/etc/passwd | wc-l how many lines are in this file?
[Root @ test/root] # cat/etc/passwd | wc-w how many words are in this file !?
• The uniq command is used to delete duplicate rows and only display one 』
Syntax: [root @ test/root] # uniq
Example: [root @ test/root] # last | cut-d ""-f1 | sort | uniq
• The tee command redirects data to a file and displays the data on the screen
Syntax: [root @ test/root] # last | tee last. list | cut-d ""-f1
Example: [root @ test/root] # last | tee last. list | cut-d ""-f1
• Tr
Syntax: [root @ test/root] # tr [-ds] SET1
Parameter description:
-D: Delete the set1 string.
-S: replace repeated characters!
Example: [root @ test/root] # Last | tr '[A-Z] ''[A-Z]' <= replace lowercase with uppercase
[Root @ test/root] # Cat/etc/passwd | tr-D: <==: this symbol is missing in/etc/passwd!
[Root @ test/root] # Cat/home/test/dostxt | tr-d' \ R'> dostxt-nom
• Split
Syntax: [root @ test/root] # split [-BL] leading character of the output file of the input file
Parameter description:
-B: file size.
-L: minute by number of rows
Example: [root @ test/root] # split-L 5/etc/passwd test <= will generate files such as testaa, testab, testac...
Note: It is much simpler under Linux! If you want to split the file, use-B size to limit the size of the split file. If it is the number of lines, use-l line to separate it!
Pipeline commands are very important in Bash's continuous processing program! In addition, Log File analysis is also an important part.
You can use the standard input parameter "-" for more careful control of the standard input for pipeline delivery to a command. For example, the cat command example
Eg: Sort mylist | more
Sort mylist | cat-N | LPR
PWD | cat-mylist | LPR