1.10 Data Flow redirection (learning process)

Introduction to Data stream redirection experiment

You may be a little unfamiliar with the concept of redirection, but you should have seen or manipulated it many times in the previous lesson > >> and know that they are directing the standard output to a file or appending it to a file, respectively. This is actually redirection, redirecting data that was originally output to standard output to a file, because the standard output ( /dev/stdout ) itself is also a file, and we have no problem with directing the output of the command to another file.

One, data flow redirection

Let's briefly review the two redirect operations we used earlier:

$ echo ‘hello shiyanlou‘ > redirect $ echo ‘www.shiyanlou.com‘ >> redirect$ cat redirect

Of course there is no problem with the previous < and << operation, as you understand, the difference is that the direction of redirection is inconsistent, that > is, from left to right, < right to left.

1. Simple redirection

Before we learn more about Linux redirection, we need to know some basic things, and we've already mentioned that Linux defaults to three special devices for terminal display and output, stdin (standard input, which corresponds to your input at the terminal), stdout (standard output, corresponding to the output of the terminal), stderr (standard error output, corresponding to the output of the terminal).

File Descriptor	Device Files	Description
0	/dev/stdin	Standard input
1	/dev/stdout	Standard output
2	/dev/stderr	Standard error

File Descriptor: The file descriptor is formally a non-negative integer. In fact, it is an index value that points to the record table in which the kernel opens a file for each process maintained by the process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In programming, some of the underlying programming often revolves around file descriptors. However, the concept of file descriptors is often applied only to operating systems such as UNIX and Linux.

There is also a symbol - , which can be used as the previous command.

We can use these file descriptors in this way:

The default is to use the terminal's standard input as the output of the command as input and standard output of the command

$ cat （按Ctrl+C退出）

REDIRECT Cat's continuous output (Heredoc mode) to a file

$ mkdir Documents$ cat > Documents/test.c\~ <<EOF#include <stdio.h>int main(){ printf("hello world\n"); return 0;}EOF

Use a file as input to the command, and standard output as the output of the command

$ cat Documents/test.c\~

The echo command passes through the data in the pipeline as input to the Cat command, using the standard output as the output of the command

echo ‘hi‘ | cat

Redirect the output of the echo command from the default standard output to a normal file

$ echo ‘hello shiyanlou‘ > redirect$ cat redirect

Beginner here to be careful not to confuse pipelines and redirects, the pipeline defaults to the input of the output of the previous command to the next command , and redirects usually require a file to establish a connection to the two commands, so you can take a closer look at the similarities and differences between the third and the last two operations.

2. Standard error redirection

REDIRECT standard output to a file, this is a very practical operation, another very useful operation is to redirect the standard error, standard output and standard errors are pointed to the screen display of pseudo-terminal, so we often see the output of a command is usually included both standard output and standard error results. For example, the following actions:

# 使用cat 命令同时读取两个文件，其中一个存在，另一个不存在$ cat Documents/test.c\~ hello.c# 你可以看到除了正确输出了前一个文件的内容，还在末尾出现了一条错误信息# 下面我们将输出重定向到一个文件，根据我们前面的经验，这里将在看不到任何输出了$ cat Documents/test.c\~ hello.c > somefile

Unfortunately, the error message is still there because, as I said, standard output and standard errors point to the terminal screen, which is not the same. There are times when we have to be able to hide certain errors or warnings, so what should we do? This will require the file descriptor that we have described earlier:

# 将标准错误重定向到标准输出，再将标准输出重定向到文件，注意要将重定向到文件写到前面$ cat Documents/test.c\~ hello.c >somefile  2>&1# 或者只用bash提供的特殊的重定向符号"&"将标准错误和标准输出同时重定向到文件$ cat Documents/test.c\~ hello.c &>somefilehell

Note that you should add the output redirection file descriptor before the & shell is redirected to a file named 1

3. Use teecommand to redirect to multiple files at the same time

Often you may have the need to print the information to the terminal in addition to redirecting the output to a file, so you can use the tee command to implement:

echo ‘hello shiyanlou‘ | tee hello

4. Permanent Redirection

You should be able to see that our previous redirection operation is only temporary, that is, only valid for the current command, how to do "permanent" effective, such as in a script, you need a part of the output of the command to redirect all, it is necessary for you to add a temporary redirect on each command of the operation, of course, do not need, We can use exec commands to implement "permanent" redirects. execThe purpose of the command is to replace the current shell with the specified command, to replace the current process with a process, or to specify a new redirect:

# 先开启一个子 Shell$ zsh# 使用exec替换当前进程的重定向，将标准输出重定向到一个文件$ exec 1>somefile# 后面你执行的命令的输出都将被重定向到文件中,直到你退出当前子shell，或取消exec的重定向（后面将告诉你怎么做）$ ls$ exit$ cat somefile

5. Create an output file descriptor

The default in the Shell can have 9 Open file descriptor, which we used is also it is provided by default, 0 the 1 2 number file descriptor, in addition we can also use the 3-8 file descriptor, but they are not open by default, you can use the following command to view the current Open file descriptor in Shell process:

cd /dev/fd/;ls -Al

Also use exec the command to create a new file descriptor:

$ zsh$ exec 3>somefile# 先进入目录，再查看，否则你可能不能得到正确的结果，然后再回到上一次的目录$ cd /dev/fd/;ls -Al;cd -# 注意下面的命令>与&之间不应该有空格，如果有空格则会出错$ echo "this is test" >&3$ cat somefile$ exit

6. Close the file descriptor

As we opened the 3rd file descriptor above, you can close it using the following actions:

exec 3>&-$ cd /dev/fd;ls -Al;cd -

7. Output of a fully shielded command

In Linux there is a device file that becomes a "black hole", so the data that is imported into it will be "swallowed".

In a UNIX-like system, a/dev/null, or empty device, is a special device file that is typically used to discard unwanted output streams or as empty files for input streams, which are usually done by redirection. Reading it will immediately get an EOF.

We can use /dev/null the output of a shielded command:

1>/dev/null 2>&1

Such an operation will result in no output from you.

8. Using Xargs to split the parameter list

Xargs is a common command for UNIX and UNIX-like operating systems. It does this by converting the argument list to small pieces to pass to other commands to avoid the problem of too long a parameter list.

This command is useful in some cases, especially if the results of commands such as find,locate and grep that produce a lot of output are used, see the man documentation for details.

-d: -f1 < /etc/passwd | sort | xargs echo

This command /etc/passwd : uses the command to generate a list after the file is sorted by the first field in the split echo .

Homework

1, understand the role of this code, the actual code will not work, please combine this section of knowledge analysis of this code does not work correctly, and try to solve the problem.

If you do not have the basics of Shell scripting, you can choose to skip or go to the Advanced Bash Programming Guide here to learn

while read filename; do  rm -iv $filenamedone <<(ls)

2, previously introduced a command line to convert the picture to ASCII character view of the tool Aview/asciiview, but it is black and white. Now, here is a color:

$ sudo apt-get install caca-utils$ cacaview <pic_file>$ cacademo$ cacafire

1.10 Data Flow redirection (learning process)