What happens when the program executes
When you double-click the Terminal program icon on the desktop, a program loaded into the shell opens. The commands you type do not execute directly in the kernel, but interact with the shell first.
Command (eg. ' Ls-l ')
↓
Terminal program (eg. ' gnome-terminal ')
↓
Shell (eg Bash)
↓
Kernel (eg. linux 2.6.24)
More information on how the process works:
When you execute a program through Python, you can choose to execute it directly from the kernel or through the shell. If you choose to execute directly, you will not be able to execute commands in the same way as bash. Www.codesec.net
Let's take a look at how to use the shell and the fun features, and then do the same with subprocess.
Data flow
Under UNIX and Linux, there are three I/O channels called streams that connect programs through text terminals (such as running Bash with gnome-terminal) and other applications such as subprocess with Python. These I/O channels are called standard inputs, standard outputs, and standard error outputs, respectively, and their file descriptors are 0,1,2.
Handle Name Description
0stdin Standard input
1stdout Standard Output
2stderr standard Error Output
Here you can see that the standard input is called stdin, the standard output is called STDOUT, and the standard error output is called stderr.
The flow works like this: getting input from the terminal output and sending it to the program via standard input, the normal output returned by the program is output from the standard output, and the error returns to the standard error output of the environment context. Wikipedia has a picture that will describe the process:
If you want to redirect a stream from one program to another, see the decomposition below.
Using the Shell
redirect standard input and output to a file
You can use the > operator in Bash to redirect the standard output of a program to a file (perhaps slightly grammatical differences in other shells). Here's an example:
Program1 > File1
The output from the program1 execution is written to file1 from the standard output stream and replaced with the existing contents of the File1. If you just want to append content, you can use the >> operator:
Program1 >> File1
The < operator can be used to read data from a file and transfer it to the standard input stream of the program:
Program1 < File1
Similarly, PROGRAM1 will be executed, but at this point File1 replaces the keyboard and becomes the standard input data source.
You can combine shell operators to accomplish more complex operations. In the following example, PROGRAM1 obtains data from file1 and sends it to standard input. The standard output is output from program1 to File2.
Program1 < file1 > File2
There may be times when you need to get the output from one program and use it as input to another program. You can do this with a temporary file:
Program1 > Tempfile1
Program2 < Tempfile1
RM Tempfile1
This method is a bit cumbersome, so the shell provides a convenient mechanism called the pipeline
Pipeline
Pipelines allow the standard output of one program to be entered directly into the standard input stream of another program without the need to create a temporary file:
program1 | Program2
operator | is called a pipe symbol, so this operation is called a pipe.
Here is a picture from Wikipedia describing the pipeline:
Here's an example of using find (traversing files and directories in the current directory) to direct the output to the GREP program to find a specific file:
Find. | grep "The file I ' m after.txt"
The first program produces data that is directed to the second program in one line, so the second program can start using them before the first program runs.
redirect standard input and output from a file
While redirecting the standard output, you can also redirect other streams, such as redirecting standard error output to standard output. We've already talked about using the >,< and >> operators to redirect the data flow before the file descriptor in Bash (remember the number 0,1,2 discussed earlier). If we omit the number 1 represented by the standard output, we will see that we have been using standard output.
The following command executes PROGRAM1 and outputs all standard error data to File1.
Program1 2881064151> File1
When the program1 is executed, the error message is redirected to file.
Here's an example program that lets you test and save it as a redirect1.py:
Import Sys
While 1:
Try
input = Sys.stdin.readline ()
If input:
Sys.stdout.write (' Echo to stdout:%s '%input)
Sys.stderr.write (' Echo to stderr:%s '%input)
Except Keyboarderror:
Sys.exit ()
This program will always accept input data and output it to stdout and stderr at the same time.
In the shell derived from csh, the syntax is followed by the redirection symbol & symbol, can achieve the same effect. (Translator note: |&)
Another common feature is the redirection of an output stream to a fixed one. The most common use is to redirect the standard error output to standard output so that the error message and the correct information can be combined, such as:
Find/-name profile > Results 2>&1
The command will find all the files called. Profile. Without redirection, it will output hit information to stdout, error messages to stderr (such as some directories without access). If the standard output is directed to a file, the error message is displayed on the command line. In order to see both the hit and error messages in the result file, we need to use 2>&1 to output the standard error output (2) to the standard output (1). (This time, even in Bash, you'll need a & character.) )
Although it is syntactically possible to put 2>&1 in front of >, this does not work correctly. In fact, when the parser reads 2>&1, it does not know where the standard output will be redirected, so the standard error output will not be merged.
If you use a pipe to merge the output stream, the merge symbol 2>&1 needs to precede the pipe symbol |. Like what:
Find/-name Profile 2>&1 | Less
The abbreviated form of the merge output in Bash is:
Command > File 2>&1
For:
Command &>file
Or:
Command >&file
But it's best not to use shorthand, or you'll get confused. I advocate rather trouble but be clear.
The &> operator redirects both standard output and standard error output at the same time. It functions like command > File 2>&1 in the Bourne Shell.
Pipe Chain
Redirects can be combined with pipelines to form complex commands, such as:
ls | grep ' \.sh ' | Sort > Shlist
List all the files in the current directory, then filter the content that contains only. sh, sort by text encoding, and then output the final result to shlist. This type of command is often used in shell scripts and batch files.
Multiple output redirection
The standard command tee can redirect a command to multiple places.
LS-LRT | Tee XYZ
This outputs the list of files to both standard output and file XYZ.
Here documentation
Most shells, including Bash, support the here document, which allows you to embed text blocks into commands using the << operator and some text as separators.
In the following example, the text block is passed to the TR command, using End_text as the here document delimiter to indicate the beginning and end of the text.
$ TR A-Z A-<<end_text
> One, three
> Uno dos tres
> End_text
One and three
UNO DOS TRES
After TR processing, the result of the output is one of the three and Uno DOS TRES.
A common use is to add text to a file with the here document. By default, the variables in the text are replaced with real values.
$ cat << EOF
> Working dir $PWD
> EOF
Working Dir/home/user
This escape can be avoided by quoting single or double quotation marks on the Here Document label:
$ cat << "EOF"
> Working dir $PWD
> EOF
Working dir $PWD
Introduction subprocess
Just now we've discussed some of the features that the command line provides, so let's try the subprocess module. You can run the following simple command from the command line:
$ echo "Hello world!"
Hello world!
Let's try to run it in Python.
Previously we needed to use a bunch of different standard libraries to implement process management. Starting with Python 2.4, all the features have been carefully collated into the subprocess module, where the Popen class can provide all we need.
Attention
If you are replacing the old module with the new Popen, [Subprocess-doc][subprocess-documentation] has a chapter explaining how the past works and how it works today.
Popen can accept the parameters, details can be in the [using-the-subprocess-module][http://docs.python.org/library/subprocess.html# Using-the-subprocess-module]:
Subprocess. Popen (args, bufsize=0, Executable=none, Stdin=none,
Stdout=none, Stderr=none, Preexec_fn=none, Close_fds=false,
Shell=false, Cwd=none, Env=none, Universal_newlines=false,
Startupinfo=none, Creationflags=0
)
Using the Shell
Let's have a Hello world! This example begins. Like before, execute the following commands through the Python shell:
>>> Import subprocess
>>> subprocess. Popen (' echo ' Hello world! "', shell=true)
Hello world!
<subprocess. Popen Object at 0x...>
As you can see, the standard output and the same print out Hello world!, the difference is that the command line shows a subprocess we created. Popen instance.
If you save the code as process_test.py and then execute it at the command line, you will get the same result:
$ python process_test.py
Hello world!
Appears to run OK.
You may be wondering which shell we are using. The default shell for Unix is/bin/sh, while under Windows it depends on the COMSPEC environment variable. If you set Shell=true, you can customize the shell with the executable parameter.
>>> subprocess. Popen (' echo ' Hello world! "', Shell=true, executable="/bin/bash ")
Hello world!
<subprocess. Popen Object at 0x...>
As we saw before, but if you use a particular shell, you may find a different place.
Let's explore the other features of using the shell with Python:
Variable resolution:
>>> subprocess. Popen (' Echo $PWD ', shell=true)
/home/james/desktop
<subprocess. Popen Object at 0x...>
Pipelines and redirects:
Subprocess. Popen (' echo ' Hello world! ' | tr A-Z-a-2> errors.txt ', shell=true)
<subprocess. Popen Object at 0x...>
>>> HELLO world!
The errors.txt should be empty because there are no errors to produce. Interestingly, on my computer, the Popen instance appears before the Hello world! is printed to the standard output. Well, pipelines and redirects work fine.
Here Documentation:
>>> subprocess. Popen ("" "
... cat << EOF > New.txt
... Hello world!
... Eof
... "" ", Shell=true)
<subprocess. Popen Object at 0xb7dbbe2c>
The New.txt file is normally generated and contains the content Hello world!.
As we expect, commands that run normally in the shell can also run in the Python shell.
String and argument list
It is now easy to execute the command line in Python, and you may need to pass the variable past. Let's say we're going to rewrite that function with Echo:
def print_string (String):
Print string
You might take it for granted:
def print_string (String):
Subprocess. Popen (' echo '%s '%string, shell=true)
This notation is not a problem when the string is Hello world!:
>>> print_string (' Hello world! ')
Hello world!
But that's a problem:
>>> print_string (' nasty ' example ')
/bin/sh:syntax error:unterminated quoted string
This command will be executed as echo "nasty" example ", well, there is a problem with escaping.
One way to do this is to make it better in your code, but it can be cumbersome, and you need to handle all the possible escape characters and spaces, and so on.
Python can help you deal with the condition that you can't directly manipulate the shell and how to do it.
Outside the Shell
Now let's try not to manipulate the shell to achieve the same effect:
def print_string (String):
Subprocess. Popen ([' echo ', string], Shell=false)
>>> print_string (' Hello world! ')
Hello world!
>>> print_string (' nasty ' example ')
Nasty "Example
Outside the shell try not to manipulate the shell to achieve the same effect