Linux I/O redirection and pipelines (about tr and tee commands)

Linux I/O redirection and pipelines (about tr and tee commands)

When I first started to get in touch with IO redirection, I had gone through the process of understanding, understanding, and then rolling. Of course, now I cannot fully understand IO redirection, I am not confused about IO redirection when solving some problems or implementing some results.

What is IO redirection?

To solve this problem, we need to first understand what is IO and what is program data stream.
What is IO?
In Linux or computer, I/O refers to the input and output of information.
Devices that can be used in Linux: files (everything in Linux is a file)

These include: keyboard devices, general files on the file system, and network cards.
Device Used for output in Linux: File
Including common files and network cards on the monitor and file system.
What is data stream?
The information we usually describe is essentially data. because the information is dynamic, we call the concepts of information or commands as flow data, that is, data stream.
Data streams are classified into InputStream and OutputStream. The input stream can only be read and not written, but the output stream can only be written and not read. Generally, the input stream is used in the program to read data, and the output stream writes data, as if the data stream is imported into the program and exported from the program. Data streams are used to separate input and output operations of programs from related devices. The input stream can obtain data from the keyboard or file, and the output stream can transmit data to the display, printer, or file.
In Linux, there are three types of program data streams:
Input data stream: standard input STDIN common device: keyboard
Output Data Stream: → standard output STDOUT common device: Terminal
Error output stream: → error output STDERR common devices: Terminals
Open a file in Linux. After the file is loaded, a number is required for the system. The number is fd: file descriptor.
Because everything in Linux is a file, the corresponding standard input device, standard output device, and error output device have their corresponding fd
Standard Input: 0
Standard output: 1
Error output: 2
IO redirection means to redirect the original default input/output path to another file.

I know it will still be confusing to explain it here. Don't worry. This concept needs to be explained through a certain amount of experiments.

 

--------------------------------------------------------------------------------
Output redirection

Common>,>,> |
>:Overwrite output >>: append output >|: overwrite forcibly
Because improper use may have a great impact on the original data, sometimes we will manually set the feature to close>. The command is:
Set-C disable overwrite output reset function
Set + C enables overwrite output redirection
However, the preceding command is invalid for "> | ".
The following describes the specific experiment content.
Lab environment: Under the/test/directory

[Root @ localhost test] # ll
Total usage 0
[Root @ localhost test] # echo $ (date)> t1
[Root @ localhost test] # ll
Total usage 4
-Rw-r --. 1 root 43 August 2 18:15 t1
[Root @ localhost test] # cat t1
Tuesday, August 02, 2016 18:15:28 CST
[Root @ localhost test] # echo test> t1
[Root @ localhost test] # cat t1
Test
[Root @ localhost test] # echo $ (who)> t1
[Root @ localhost test] # cat t1
Test
Root: 0 2016-08-02 (: 0) root pts/0 2016-08-02 (192.168.85.1) root pts/1 2016-08-02 19 (192.168.85.1)
[Root @ localhost test] # set-C
[Root @ localhost test] # echo 111> t1
-Bash: t1: The existing files cannot be overwritten.
[Root @ localhost test] # echo 111> | t1
[Root @ localhost test] # cat t1
111

The preceding results show the functions of >>>>and >|.

I believe we have a better understanding of input redirection.

 

--------------------------------------------------------------------------------
Error output stream redirection
2>

[Root @ localhost test] # echoo 1> t2
Bash: echoo: Command not found...

When a command is incorrect, shell returns an error message. If you want to save the error message to a file, you can perform the following operations:
[Root @ localhost test] # echoo 1 2> t2
-Bash: t2: existing files cannot be overwritten.
[Root @ localhost test] # ll
Total usage 4
-Rw-r --. 1 root 4 August 2 18:18 t1
-Rw-r --. 1 root 0 August 2 18:20 t2
[Root @ localhost test] # echoo 1 2> t3
[Root @ localhost test] # cat t3
Bash: echoo: Command not found...
[Root @ localhost test] # echop wqew 2> t3
[Root @ localhost test] # cat t3
Bash: echoo: Command not found...
Similar commands: 'echo'
Bash: echop: Command not found...

Error redirection also supports append.
If you want to record data regardless of whether the command is correct or not, you can use the following format:
1. & >,&>
[Root @ localhost test] # ech2 &> t4
[Root @ localhost test] # echo right &> t4
[Root @ localhost test] # cat t4
Bash: ech2: Command not found...
Right

2. command>/path/somewhere... 2> & 1 The second format can only be set to 2> & 1 regardless of whether append or overwrite. 2 >>& 1 indicates the error format. If it is used, an error is reported in CentOS7.2.
[Root @ localhost test] # ech2> t5 2> & 1
[Root @ localhost test] # echo right> t5 2> & 1
[Root @ localhost test] # cat t5
Bash: ech2: Command not found...
Right

Note: The output redirected object can be a variety of files, but it cannot be directed to the command input. To implement this function, you need to use pipelines. This may be a bit confusing. The following is an example

[Root @ localhost test] # who> tr a-z A-Z> t6
[Root @ localhost test] # cat t6
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
[Root @ localhost test] # ll
Total usage 4
-Rw-r --. 1 root 54 Aug 17 18:53 t6
-Rw-r --. 1 root 0 August 2 18:53 tr

We will redirect the command displayed by who to the input of the tr command, convert the content in lower case to upper case, and then redirect to the t6 file. What we want to achieve is this result, but it is not implemented. Instead, the tr file appears in the process. This is because in the first-layer redirection, tr is used as the file name by default, But the strange thing is that there is nothing in tr, so the understanding of this error can be removed as soon as possible.
[Root @ localhost test] # cat tr
[Root @ localhost test] #

We can use pipelines to perfectly solve the above problems.
[Root @ localhost test] # who | tra-z A-Z> t6
[Root @ localhost test] # cat t6
ROOT: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)

What is pipeline? I believe that through the above example, you should have a preliminary understanding of it.

Pipeline: A Connection Program that directly directs the output of the previous command to the next program as an input data stream.
The commonly used expression format is as follows: COMMAND1 | COMMAND2 | COMMAND3 |...
It is a supplement and enhancement of the output redirection, and its essence is still the class which belongs to the output redirection.
Now that we are talking about pipelines, we can add more here.
Commands commonly used in pipelines:
Tee
Tee-read from standard input and write to standard output and files
Display standard input results and write them to files
Tee format:
Command 1 | tee file name | command 2
Save the STDOUT of command 1 in the file name, and then input the pipe to command 2
Purpose:
Save the output of Different Stages
Troubleshooting of complex Pipelines
View and record output simultaneously
Here is a simple example to illustrate the specific role of tee.
[Root @ localhost test] # who | tee t2 | tr a-z A-Z
ROOT: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)
[Root @ localhost test] # cat t2
Root: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)

Tee records the previous result in the t2 file and displays the result of the previous command.

--------------------------------------------------------------------------------
Input redirection
<Single-line mode, <multi-line mode

To learn about input redirection, we must first understand a command
Tr
Tr-translate or delete characters convert or delete characters
The expression format is as follows:
Tr [OPTION]... SET1 [SET2]
Option:
-C or -- complerment: obtains the complement of the character set.
-D or -- delete: delete all characters belonging to the first character set
-S or-squeeze-repeats: represents consecutive duplicate characters with a separate character
Next, let's take an example to briefly understand the role of tr.

12345678 [root @ localhost test] # cat t1
Root: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)
[Root @ localhost test] # tr a-z A-Z <t1
ROOT: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)

T1 is the content of who stored by tee. We use input redirection to input tr and convert it from lowercase to uppercase.
Input redirection is to replace the content originally entered by the keyboard with other text content or file content.
I think you may not understand <multiline redirection. Here I will explain the multiline redirection in the example.
12345678 [root @ localhost ~] # Tr a-z A-Z
> Morning, boys
> Today is another day
> Please be happy
> End
MORNING, BOYS
TODAY IS ANOTHER DAY
PLEASE BE HAPPY

For multiline input redirection, the end string must be followed by the specified ending string. Generally, eof or end is used as the ending identifier.
For multiline input redirection, you must manually enter the content. To redirect a single line of input, you must end it with the output of the file or the previous command.

--------------------------------------------------------------------------------
I know that there will still be some crashes here. The following nine instances are used to combine the standard output input and pipeline commands.

Lab environment CentOS7.2

1. Convert the content in the/etc/issue file to uppercase and save it to the/test/issue # file.
1234567 [root @ localhost test] # cat/etc/issue | tr a-z A-Z>/test/issue1
[Root @ localhost test] # echo $ (tr a-z A-Z </etc/issue)>/test/issue2
[Root @ localhost test] # cat issue1
\ S
KERNEL \ r on an \ M
[Root @ localhost test] # cat issue2
\ S kernel \ r on an \ M

The above two methods can both achieve the requirements, but the output format is slightly different, this command is related. There are no further restrictions on the two expressions. First, use pipelines, output redirection, and second, use command reference, input redirection, and output redirection.

2. Convert the login user information of the current system into uppercase and save it to the/test/t2 file.
12345 [root @ localhost test] # who | tr a-z A-Z>/test/t2
[Root @ localhost test] # cat t2
ROOT: 0 2016-08-02 10: 12 (: 0)
Root pts/0 2016-08-02 10: 12 (192.168.85.1)
Root pts/1 2016-08-02 10: 19 (192.168.85.1)

3. A linux user sends an email to the root user. The email must be titled "help". The body of the email is as follows:
Hello, I am user name, the system version is here, pleasehelp me to check it, thanks!
Operating system version information
We use user gentoo
[Root @ localhost ~] # Su gentoo
[Gentoo @ localhost/root] $ whoami
Gentoo
[Gentoo @ localhost/root] $ cd/home/gentoo/
[Gentoo @ localhost gentoo] $ echo-e "Hello, I am 'whoam' \ nThe system version is here, please help me to check it, thanks. \ nThe version is 'cat/etc/RedHat-release '"| mail-s" help "root
[Gentoo @ localhost gentoo] $ exit
Exit
You have a new email in/var/spool/mail/root.
[Root @ localhost ~] # Mail
Heirloom Mail version 12.5 7/5/10. Type? For help.
"/Var/spool/mail/root": 3 messages 1 new
1 gentoo@localhost.loc Tue Aug 2 21/731 "help"
2 root Tue Aug 2 21/741 "help"
> N 3 gentoo@localhost.loc Tue Aug 2 20/737 "help"
& 3
Message 3:
From gentoo@localhost.localdomain Tue Aug 2 19:37:36 2016
Return-Path: <gentoo@localhost.localdomain>
X-Original-To: root
Delivered-To: root@localhost.localdomain
Date: Tue, 02 Aug 2016 19:37:36 + 0800
To: root@localhost.localdomain
Subject: help
User-Agent: Heirloom mailx 12.5 7/5/10
Content-Type: text/plain; charset = us-ascii
From: gentoo@localhost.localdomain
Status: R
-E Hello, I am gentoo
The system version is here, please help me to check it, thanks.
The version is CentOS Linux release 7.2.1511 (Core)

Here, the command replacement is used to display the user and version information. The pipeline passes the output of the first command to the next command as its input.

4. display the/root/file list in one row and separate the file names with spaces.
[Root @ localhost test] # ls-1/root | tr '\ n' \ n
Anaconda-ks.cfg CST dead. letter file initial-setup-ks.cfg test.txt tr public template Video Image file Download music desktop [root @ localhost test] #

The above command has met our requirements, but it looks awkward without line breaks. Below we will make the output more comfortable with perfection.
[Root @ localhost test] # echo-e 'LS-1/root/| tr' \ n' \ N'
Anaconda-ks.cfg CST dead. letter file initial-setup-ks.cfg test.txt tr public template video image documentation download music Desktop
[Root @ localhost test] #

Use the echo-e to enable the escape character to enable the newline \ n.

5. The content of the file1 file is: "1 2 3 4 5 6 7 8 9 10" to calculate the sum of all numbers.
[Root @ localhost test] # cat> file1
1 2 3 4 5 6 7 8 9 10
^ C
[Root @ localhost test] # cat file1
1 2 3 4 5 6 7 8 9 10

Create the file1 folder and enter the required content.
12 [root @ localhost test] # cat file1 | tr ''' + '| bc
55

6. process the string "xt., l 1 jr #! $ Fe2 c */fe3 uz4 ", only the numbers and spaces
[Root @ localhost test] # echo 'xt., l 1 jr #! $ Fe2 c */fe3 uz4 '| tr-cd' [: digit:] [: space:]'
1 2 3 4

Here we use-c and-d supplements and deletions in tr. flexible use of the supplements can help us filter unwanted information more accurately.

7. display each directory of the PATH variable in an independent line.
[Root @ localhost test] # echo $ {PATH}
/Usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
[Root @ localhost test] # echo $ {PATH} | tr ': ''\ N'
/Usr/local/sbin
/Usr/local/bin
/Usr/sbin
/Usr/bin
/Root/bin

This is a simple question. You only need to display the PATH correctly. Use tr to replace ":" With a line break.

8. Delete empty lines of the specified file
Here we create a text file t9 that contains many blank lines.
[Root @ localhost test] # cat t9
Q



Ere
S



Df
Sd




Sd
[Root @ localhost test] # cat t9 | tr-s '\ N'
Q
Ere
S
Df
Sd
Sd

Here, the option "Duplicate Delete-s" of tr is used.

9. Each word (letter) in the file is displayed on an independent line without blank lines.
Here we select the/etc/rc. d/rc0.d/K50netconsole file, and use the head to view the content of the first five lines of the file.
[Root @ localhost test] # head-5/etc/rc. d/rc0.d/K50netconsole
#! /Bin/bash
#
# Netconsole This loads the netconsole module with the configured parameters.
#
# Chkconfig:-50 50
[Root @ localhost test] # cat/etc/rc. d/rc0.d/K50netconsole | tr-cs '[: alpha:] ''\ n' | head-5

Bin
Bash
Netconsole
This

The results show that we have implemented the requirements of the question, but why is the first line empty? This is because-s compresses all empty rows into one row, so there will be a blank row in front.

This article permanently updates the link address: