Python's subprocess module (ii)

Original: http://blog.chinaunix.net/uid-26000296-id-4461522.html

Introduction of Subprocess Module

The subprocess was first introduced in version 2.4.
The subprocess module is used to generate sub-processes and can be piped to connect their input/output/errors, and to obtain their return values.
It is used to replace multiple old modules and functions:
Os.system
os.spawn*
os.popen*
popen2.*
commands.*

The old functions that can be substituted for this module can be found in the Subprocess-replacements section.
POSIX users (Linux, BSD, etc) can also install and use the updated SUBPROCESS32 module instead of the subprocess in Python version 2.7.
Although SUBPROCESS32 is a low version, it works better in some cases.

1.1. Using the Subprocess module

The recommended way to start a subprocess is to use the following convenience features.
When these are not sufficient, the underlying Popen interface needs to be used.

1. Subprocess.call

Grammar:
Subprocess.call (args, *, Stdin=none, Stdout=none, Stderr=none, Shell=false)
Semantic:
Runs the command specified by args until the command ends, returning the property value of the return code.

The above parameters are the most common way, the following is the sample code:
>>>
>>> subprocess.call (["LS", "-l"])
0
>>> Subprocess.call ("Exit 1", shell=true)
1
WARNING: Using Shell=true is a security protection mechanism.
Note: When using this function, do not use the Stdout=pipe or stderr=pipe parameters,
Otherwise, it causes a deadlock to output the child process.
If you want to use a pipe, you can use the Popen in the Communicate () method
Example code:
Import subprocess
rc = Subprocess.call (["LS", "-l"])

A shell can be used to interpret an entire string:
Import subprocess
out = Subprocess.call ("Ls-l", Shell=true)
out = Subprocess.call ("CD..", Shell=true)

This parameter is used for shell=true.
At this point, we use an entire string instead of a table to run the child process.
Python will run a shell first and then use this shell to interpret the entire string.

Some of the shell commands are built-in shell commands that must be run through the shell, $CD.
Shell=true allows us to run some of these commands.

2. Subprocess.check_call

Grammar:
Subprocess.check_call (args, *, Stdin=none, Stdout=none, Stderr=none, Shell=false)
Semantic:
Runs the command specified by args until the command execution is complete.
If the return code is zero, it is returned. Otherwise, the Calledprocesserror exception is thrown.
The Calledprocesserror object contains a property value with a return code.

The parameters shown above are only the most common, and the following are the more commonly used parameters for the user.
The sample code is as follows:
>>>
>>> subprocess.check_call (["LS", "-l"])
0
>>> Subprocess.check_call ("Exit 1", shell=true)
Traceback (most recent):
...
Subprocess. Calledprocesserror:command ' exit 1 ' returned Non-zero exit status 1

This function was introduced in Python version 2.5.
WARNING: Using Shell=true is a security mechanism.
Note: Do not use Stdout=pipe or stderr=pipe in this function, otherwise it will cause the child process to deadlock.
If you need to use a pipe, you can use Popen in the Communicate () method.

3. Subprocess.check_output

Grammar:
Subprocess.check_output (args, *, Stdin=none, Stderr=none, Shell=false, Universal_newlines=false)
Semantic:
Runs the args-defined command and returns a string representing the output value.
If the return code is nonzero, the Calledprocesserror exception is thrown.
Example code:
>>>
>>> subprocess.check_output (["echo", "Hello world!"])
' Hello world!\n '
>>> Subprocess.check_output ("Exit 1", shell=true)
Traceback (most recent):
...
Subprocess. Calledprocesserror:command ' exit 1 ' returned Non-zero exit status 1
If you want to capture standard errors in the results, use stderr=subprocess. StdOut parameters:
>>>
>>> Subprocess.check_output (
... "LS Non_existent_file; Exit 0 ",
... stderr=subprocess. STDOUT,
... shell=true)
' Ls:non_existent_file:No such file or directory\n '
This function was introduced in Python version 2.7.
WARNING: Using Shell=true is a security mechanism.
Note: Do not use Stdout=pipe or stderr=pipe in this function, otherwise it will cause the child process to deadlock.
If you need to use a pipe, you can use Popen in the Communicate () method.

4. subprocess. PIPE

When using Popen, special values for stdin, stdout, and stderr parameters are used to indicate that the pipe that connects the standard stream is opened.

5. subprocess. STDOUT

When using Popen, a special value for the stderr parameter indicates that a standard error is redirected to the same handle of the standard output.

6. Abnormal subprocess. Calledprocesserror

The exception that is thrown when a process running by Check_call () or Check_output () returns a non-0 status value.

7. ReturnCode

The exit status of the child process.

8. cmd

The command that the child process executes.

9. Output

If Check_output () throws an exception, the output value of the child process.
Otherwise, this value is not.

1.1.1. Common parameters

To support various user usage scenarios, the Popen build function receives a variety of optional parameters.
For the most typical case, many parameters retain a safe default value, which is most commonly used in the following ways:

1. Args

All functions require this parameter, and it is a string, or a sequence of arguments to the program.
Providing a parameter sequence is a more recommended approach because it allows the module to receive parameters in spaces or quotes.
If a single string is passed, either shell=true, or both the string is the name of the program, and the parameter cannot be taken.

2. stdin, stdout and stderr

stdin, stdout, and stderr specify a file handle for the standard input, standard output, and standard error of the executing program.
Their values can be pipe, an existing file descriptor (a positive integer), an existing file object, or None.
Pipe represents the creation of a new pipeline that connects child processes.
The default value is None, which means no redirection is done.
The file handle of the child process can be inherited from the parent process.
In addition, stderr can set a value of STDOUT, which indicates that the error data for the child process can be the same file handle as the standard output.

When the value of stdout or stderr is pipe and the value of Universal_newlines is true,
For new lines opened with the ' U ' mode parameter, the end of all rows will be converted to ' \ n '.

3. Shell

If the shell's value is True, the specified command line is executed through the shell.
This is useful if you use Python as a process control, because it provides the vast majority of system shell commands and makes it easy to use
Shell's various functions, such as Shell pipeline, file name wildcard, environment variable extension, and user directory extension ~.
However, it is important to note that Python provides a similar implementation of shell functionality.

WARNING: Executing a shell command from an untrusted source can be a serious security issue.
Based on this, Shell=true is not recommended.
The sample code is as follows:
>>>
>>> from subprocess Import call
>>> filename = input ("What file would do like to display?\n")
What file would do you like to display?
Non_existent; RM-RF/#
>>> Call ("cat" + filename, shell=true) # Uh-oh. This'll end badly ...

Shell=false shuts down all of the shell's basic functions, so there is no security vulnerability described above.
You can see in the Help documentation for the Popen build function that it works only when you are shell=false.
When using Shell=true, Pipes.quote () can be used to translate spaces, shell characters, and so on.

1.1.2. Popen Build function

Lower-level process creation and management in subprocess can be implemented through the Popen class.
It provides more flexibility, and programmers can handle more complex situations through it.
Grammar:
Class subprocess. Popen (args, bufsize=0, Executable=none,
Stdin=none, Stdout=none, Stderr=none,
Preexec_fn=none, Close_fds=false, Shell=false, Cwd=none, Env=none,
Universal_newlines=false, Startupinfo=none, creationflags=0)
Semantic:
Executes a sub-program in a new process.
In Unix, this class uses a OS.EXECVP ()-like method to execute subroutines.
In Windows, this class uses the CreateProcess () function of Windows to execute subroutines.
Parameter resolution:
Args: A sequence of program arguments, or a single string.
By default, the program to be executed should be the first field in the sequence.
If a single string, its resolution depends on the platform
In Unix, if Args is a string, the string is interpreted as the name or path of the executed program.
However, this situation can only be used in programs that do not require parameters.

Note: Shlex.split () is useful when the correct delimiter is determined for args, especially in complex cases:
>>>
>>> Import Shlex, subprocess
>>> command_line = Raw_input ()
/bin/vikings-input eggs.txt-output "spam spam.txt"-cmd "Echo ' $MONEY '"
>>> args = Shlex.split (command_line)
>>> Print args
['/bin/vikings ', '-input ', ' eggs.txt ', '-output ', ' spam spam.txt ', '-cmd ', ' echo ' $MONEY ']
>>> p = subprocess. Popen (args) # success!

Note: Options (such as-input) and parameters (such as eggs.txt) are separated into separate list elements in the shell by spaces.
If the arguments require quotation marks or backslashes, they are a single list element.
The shell parameter (the default value of false) declares whether the shell is used to execute the program.
If shell=true, it considers args to be a string, not a sequence.

In Unix systems, and shell=true, the shell uses/bin/sh by default.
If Args is a string, it declares a command executed through the shell. This means that the string must use the correct format.
If args is a sequence, the first element is the command string, and the other elements are used as parameters.
It can be said that Popen is equivalent to:
Popen (['/bin/sh ', '-C ', args[0], args[1], ...])
BufSize: If a value is specified, it has the same meaning as the parameter corresponding to the built-in function open ():
0--Indicates no buffering
1--Indicates buffering
Any other positive value indicates the size of the buffer.
A negative value indicates that the system default value is used, which generally means full buffering.
Its default value is zero.
Note: If you experience performance issues, it is recommended that you set BufSize to 1 or a positive number that is large enough (such as 4096).

Executable: Specifies the program to use in place of execution. It is rarely used.
stdin, stdout, stderr: A file handle that specifies the standard input, standard output, and standard error of the executing program.
Valid values can be PIPE, an existing file descriptor, or the existence of a file object, or None.
The default value is None.
The stderr can be set to stdout, which indicates that the stderr data of the child process is redirected to stdout.
PREEXEC_FN: If it is set to a callable object, then this object will be called by the child process before it is executed, only for UNIX.
Close_fds: If set to True, all file descriptors except 0,1 and 2 will be closed for UNIX until the child process is executed.
CWD: When it is not none, the current path of the subroutine is replaced with the value of CWD before execution.
This path is not added to the executable's search path, so CWD cannot be a relative path.
ENV: When it is not none, it is a mapping of the environment variables of the new process.
It can be used in place of the current process environment.
Universal_newlines: When True, file objects stdout and stderr are opened as text files

Example code:
1. After the Popen object is created, the main program does not automatically wait for the child process to complete.
We must call the object's Wait () method before the parent process waits (that is, block block):
Import subprocess
Child = subprocess. Popen (["Ping", "-C", "5", "www.google.com"])
Print ("Parent process")

As seen from the results of the run, the parent process does not wait for the child to complete after it has been opened, but runs print directly.

2. Compare the waiting conditions:
Import subprocess
Child = subprocess. Popen (["Ping", "-C", "5", "www.google.com"])
Child.wait ()
Print ("Parent process")

In addition, you can also perform other operations on the child process in the parent process, such as in our example:
Child.poll () # Check child process status
Child.kill () # terminating child process
Child.send_signal () # Send a signal to a child process
Child.terminate () # terminating child process
The PID of the child process is stored in the Child.pid

3. You can change standard input, standard output, and standard error when Popen () is set up for a sub-process.
and can use subprocess. Pipes connect the inputs and outputs of multiple sub-processes together to form a pipeline (pipe):
Import subprocess
Child1 = subprocess. Popen (["LS", "-l"], stdout=subprocess. PIPE)
Child2 = subprocess. Popen (["WC"], stdin=child1.stdout,stdout=subprocess. PIPE)
out = Child2.communicate ()
Print (out)

Subprocess. Pipe actually provides a buffer for text flow.
The child1 stdout the text out to the buffer, and then Child2 's stdin reads the text from the pipe.
The output text of the child2 is also stored in the pipe until the communicate () method reads the text from the pipe.
Note that communicate () is a method of the Popen object that blocks the parent process until the child process finishes.

4. You can also use the communicate () method to input a child process using a pipe:
Import subprocess
Child = subprocess. Popen (["Cat"], stdin=subprocess. PIPE)
Child.communicate ("Vamei")
After we start the subprocess, cat waits for input until we enter "Vamei" with communicate ().

By using the subprocess package, we can run external programs. This greatly expands the functionality of Python.
If you already know some of the operating system's applications, you can call the app directly from Python (rather than relying entirely on python).
and output the results of the application to Python and let Python continue processing.
Shell functions, such as using text streams to connect to individual applications, can be implemented in Python.

1.1.3. Exceptions

The exception that is thrown by the child process is re-thrown to the parent process before the new program is started.
In addition, the exception object will have an additional property called Child_traceback, which is a string containing information traced from the point of view of the subroutine.

The most common thrown exception is OSError, when it happens, usually we execute a nonexistent file. The application should be able to handle this exception.
If you call Popen with an invalid argument, the ValueError exception is thrown.
If the return code of the called process is nonzero, check_call () and Check_output () throw the calledprocesserror exception.

1.1.4. Security

Unlike some other popen functions, this implementation would never call a system shell implicitly.
This means, characters, including Shell metacharacters, can safely is passed to child processes.
Obviously, if the shell is invoked explicitly and then it's the application ' s responsibility to ensure that
All whitespace and metacharacters is quoted appropriately.

Python's subprocess module (ii)