Thoroughly understand the advanced I/O redirection of the shell

This article directory:
1.1 Document Descriptor (file DESCRIPTION,FD)
1.2 Copying of file descriptors
1.3 Redirect Order is important: ">file 2>&1" and "2>&1 >file"
1.4 Changing the redirect target for the current shell environment
1.5 Closing file descriptors
1.6 Open File
1.7 Movement of file descriptors
1.8 Classic Examples

The basic redirection function must be understood. This article on the shell Environment of IO redirect a little deeper, I believe after reading, can thoroughly understand >file 2>&1 .

1.1 Document Descriptor (file DESCRIPTION,FD)

The file descriptor is an important concept in IO redirection. The file descriptor uses a numeric representation, which indicates the flow characteristics of the data.

Software Design believes that the program should have a data source, data export and error reporting places. In Linux systems, they use descriptors 0, 1, and two, respectively, these 3 descriptor default target files (devices) are/dev/stdin,/dev/stdout,/dev/stderr, respectively, each terminal character device is a soft link.

[[email protected] ~]# ll /dev/std*
lrwxrwxrwx 1 root root 15 Apr  2 07:57 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root 15 Apr  2 07:57 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root 15 Apr  2 07:57 /dev/stdout -> /proc/self/fd/1

[[email protected] ~]# ll /proc/self/fd/
total 0
lrwx------ 1 root root 64 Apr  6 03:53 0 -> /dev/pts/2
lrwx------ 1 root root 64 Apr  6 03:53 1 -> /dev/pts/2
lrwx------ 1 root root 64 Apr  6 03:53 2 -> /dev/pts/2
lr-x------ 1 root root 64 Apr  6 03:53 3 -> /proc/14038/fd

In Linux, each process is automatically fetched with 3 file descriptors 0, 1, and 2, respectively, for standard input, standard output, and standard error, and the file descriptor must be identified starting from 3 if you want to open another file. For us to open the descriptor, it is recommended to use a descriptor of less than 9, more than 9 of the descriptor may have been assigned to other processes within the system.

The file descriptor is a number that the system assigns to it to keep track of the open file, and the number is bound to the file, and the data flows to the descriptor, which also indicates the incoming file.

All things in Linux are files, and these files can be assigned descriptors, including sockets.

When the program opens the file descriptor, there are three possible behaviors: read from the descriptor, write to the descriptor, and be readable and writable. From the FD column of lsof, it can be seen whether the program opened the file to read data from it, to write data to it, or to read and write. For example, when the tail command monitors a file, it opens the file from which to read the data (3r R is Read,w is Write,u is read and write).

 
[[email protected] ~]# lsof -n | grep "/a.sh" | column -t                 
tail 13563 root  3r  REG 8,2 182 69632966 /root/a.sh

1.2 Copying of file descriptors (duplicate)

The copy of the file descriptor indicates that the file descriptor is copied to another file descriptor for the copy file. Use "&" for replication. If you do not understand the meaning of replication, it is understood as "binding", "reuse".

[n] <& word: Bind file descriptor n to the file or descriptor represented by word. Can be understood as the file descriptor n reuses the file or descriptor represented by word. If n is not specified, it defaults to 0 (standard input is 0), which means that standard input will also be input into the file or descriptor represented by word.

[n]> & word: Bind file descriptor n to the file or descriptor represented by word. Can be understood as the file descriptor n reuses the file or descriptor represented by word. If n is not specified, it defaults to 1 (standard output is 1), which means that standard output will also be output to the file or descriptor represented by word.

Note that each time you bind, you will be redirected to the corresponding file immediately . Please combine the following examples to feel.

For example, 3>&1 means that fd=3 is bound to fd=1, and Fd=1 's current redirect target file is/dev/stdout, so fd=3 is also redirected to/dev/stdout, Later, when the process writes data to fd=3, it will be output directly to the screen. If you use "copy" to understand, that is, fd=3 is a copy of the current fd=1, that is, pointing to the/dev/stdout device. If the fd=1 's output target (such as File1) is changed later, because Fd=3 's target is still/dev/stdout, you can take fd=3 to restore the fd=1 so that its target reverts to/dev/stdout.

For example, cat <&1 means that fd=0 is bound to fd=1, and Fd=1 's redirect file is/dev/stdout at this point, so/dev/stdout is both a standard input device and a standard output device. This means that the process accepts input from the/dev/stdout (screen) and then outputs it directly to/dev/stdout. Here are the results:

[[email protected] ~] # cat <& 1
q # Enter interactive, enter data
q # direct output

1.3 Redirect Order is important: ">file 2>&1" and "2>&1 >file"

Presumably many people know the role of >file 2>&1 , which is equivalent to &>file , which indicates that both standard output and standard errors are redirected to file. What difference does it have with the 2>&1 >file ?

First explain >file 2>&1 . Here are two procedures: first open file, and then redirect fd=1 to file files, so that the file is the output target of the standard output, then the fd=2 binding (note that the binding is not redirected) to Fd=1, and Fd=1 is now redirected to the file files, so the FD The =2 is also redirected to file. So, the end result is that the standard output is redirected to file, and the standard error is redirected to file.

Explain again 2>&1 >file . There are two processes here: Bind the fd=2 to fd=1 first, and fd=1 the redirected file is the default/dev/stdout, so fd=2 redirects to/dev/stdout, and then redirects fd=1 to the file (note that the binding is not redirection). In other words, the standard error and standard output here are still output separately, but instead of the/dev/stderr using/dev/stdout, the/dev/stdout is replaced with file. So, the end result is a standard error output to/dev/stdout, which is on the screen, and the standard output is output to the file.

You can use the following command to test 2>&1 >file . The first LS command is correct, the result is output to/tmp/a.log, the second LS command is wrong, and the result is output directly to the screen.

 
[[email protected] ~]# ls /boot 2>&1 >/tmp/a.log
[[email protected] ~]# ls sjdfk 2>&1 >/tmp/a.log
ls: cannot access sjdfk: No such file or directory

Finally, you may have discovered that bindings and redirects are different, and bindings should not be called redirects . distinguishing between these two concepts can solve a lot of doubts in the actual application process. In the last example at the end of this article, you will have a very clear understanding of the differences between bindings and redirects.

1.4 Changing the redirect target for the current shell environment

If the redirect location is changed directly in the command, the descriptor is automatically restored when the command is executed at the end. As above the ls/boot 2>&1 >/tmp/a.log command, after the LS execution is finished, fd=2 reverts back to the default/dev/stderr,fd=1 restore back to the default /dev/stdout.

However, we can change the redirect target directly in the current shell environment through the exec program, which releases the binding of the descriptor only when the current shell exits.

For example, the following command redirects standard error fd=2 to the fd=3 corresponding file.

2>&3

Therefore, we may need to restore the descriptor to its original location after the execution of a program, and close the descriptor that is no longer needed. After all, descriptors are also resources, which are limited (ulimit-n).

1.5 Closing file descriptors

[n]>&-
[n]<&-

The way to close the file descriptor is to use the symbol "-" in [N]>&word and [N]<&word ], which means releasing the Fd=n descriptor and closing the file it points to.

1.6 Open File

[n]<> filename : opens the filename and specifies that its file descriptor is N, which is a readable, writable descriptor. If n is not specified, the default is 0, and if the filename file does not exist, the filename file is created first.

For example:

 
[[email protected] ~]# exec 3<> /tmp/a.log
[[email protected] ~]# lsof -n | grep "/a.log" | column -t 
bash 13637 root 3u REG 8,2 292018 69632965 /tmp/a.log

If exec 1>&3 binds fd=0 to fd=3, then/tmp/a.log is the source of the standard input.

1.7 Movement of file descriptors

The move representation of the file descriptor moves the file descriptor 1 to the Descriptor 2 and closes the file descriptor 1.

[n]>&digit- : moves the output file represented by the file descriptor digit to n and closes the descriptor for the digit value.

[n]<&digit- : moves the input file represented by the file descriptor digit to n and closes the descriptor for the digit value.

For example:

[[email protected] ~] # exec 3 <> /tmp/a.log
[[email protected] ~] # lsof -n | grep "/a.log" | column -t
bash 13637 root 3u REG 8,2 292018 69632965 /tmp/a.log
[[email protected] ~] # exec 1> & 3- # Move 3 to 1, close 3
[[email protected] ~] # lsof -n | grep "/a.log" | column -t # View in another bash window
bash 13637 root 1u REG 8,2 292018 69632965 /tmp/a.log

As can be seen, the/tmp/a.log associated with fd=3 has been associated to fd=1 after Fd=3 moved to Fd=1.

1.8 Classic Examples

(1). Example one:

The following is an example from the Advanced Bash-scripting Guide:

echo 1234567890> File # (1). Write a string to "File".
exec 3 <> File # (2). Open "File" and assign it fd 3.
read -n 4 <& 3 # (3). Read-only 4 characters.
echo -n.> & 3 # (4). Write a decimal point.
exec 3> &-# (5). Close fd 3.
cat File # (6) .1234.67890

(1) write several characters to the file.

(2) Open file files for Read/write, and assign fd=3 to the file.

(3) The fd=0 is bound to fd=3, and the redirect target of fd=3 is file, so the target of Fd=0 is also file, that is, reading data from file. This reads 4 characters, because the Read command does not specify a variable, so it is assigned to the default variable reply. Note that after the execution of this command is complete, the Fd=0 redirect target will be changed back to/dev/stdin.

(4) The fd=1 is bound to fd=3, and the redirected target file for fd=3 is files, so the target of fd=1 is also file, that is, the data is written to file. Write a decimal point here. Note that after this command is finished, the redirect target of Fd=1 returns back to/dev/stdout.

(5) Close the fd=3, which also closes the file to which it points.

(6) A decimal is already written to the file. If the echo $REPLY is executed at this time, "1234" is output.

(2). Example two: About descriptor recovery, closing

exec 6>&1 # (1)
exec > /tmp/file.txt        # (2)
echo "---------------" # (3)
exec 1>&6 6>&-              # (4)
echo "===============" # (5)

(1) First bind the fd=6 to Fd=1, at which point the redirect target for fd=1 is/dev/stdout, so the fd=6 redirect target is/dev/stdout.

(2) redirect The fd=1 to the/tmp/file.txt file. All standard outputs will then be written to the/tmp/file.txt.

(3) write the data. The data is written to the/tmp/file.txt.

(4) Bind fd=1 back to fd=6, at which point the redirect target for fd=6 is/dev/stdout, so fd=1 will revert to/dev/stdout. Finally, the fd=6 is closed.

(5) Write the data, the data will be output on the screen.

Perhaps you will wonder why you should first bind the fd=1 to fd=6, and then use fd=6 to restore the fd=1, restore the fd=1 redirected back to/dev/stdout it is not OK?

In fact, this transfer descriptor is borrowed from fd=6 for ease of operation. You can not do it, but when you restore the Fd=1 redirect target, you should redirect to the/dev/{pseudo-terminal character device} instead of/dev/stdout, because/dev/stdout is a soft link with a target of/PROC/SELF/FD/1, but the file is still a soft link , which points to the/dev/{pseudo-terminal character device}. Similarly/dev/stdin and/dev/stderr are the same.

Therefore, if your current terminal is PTS/2, you can use the following command to achieve the same functionality as above:

exec > /tmp/file.txt
echo "---------------"
exec >/dev/pts/2
echo "==============="

If you do not borrow fd=6 this transit descriptor, it is inconvenient to first get and remember the terminal where the current shell is located. And, if it's not fd={0,1,2}, it's more of a hassle.

Finally, a process example diagram for copying and recovering a descriptor is given:

Back to Linux series article outline: http://www.cnblogs.com/f-ck-need-u/p/7048359.html
Back to Site Architecture series article outline: http://www.cnblogs.com/f-ck-need-u/p/7576137.html
Back to Database series article outline: http://www.cnblogs.com/f-ck-need-u/p/7586194.html
Reprint Please specify source: http://www.cnblogs.com/f-ck-need-u/p/8727401.html

Note: If you think this article is not bad please click on the lower right corner of the recommendation, your support can inspire the author more enthusiasm for writing, thank you very much!

Thoroughly understand the advanced I/O redirection of the shell