[Reprint] subprocess in Python

Original article address:Subprocess in Python Author:Grinding From: http://li2z.cn/2010/04/14/python_subprocess/

Just like python built-in functions, the content is all from the official Python documentation, but it is not a simple translation. Therefore, if I understand it incorrectly, please correct me. Thank you.

You can useSubprocessThis module is used to generate sub-processes and connect them to the standard input, output, and error of sub-processes. The returned values of sub-processes can also be obtained.SubprocessIt is intended to replace several other old modules or functions, such:

OS. System
OS. Spawn *
OS. popen *
Popen2 .*
Commands .*

The following describes how to useSubprocessTo replace these functions or modules.

Use subprocess Module

This module defines a class: popen

Class Subprocess.Popen(ARGs,Bufsize = 0,Executable = none,Stdin = none,Stdout = none,Stderr = none,Preexec_fn = none,Close_fds = false,Shell = false,CWD = none,ENV = none,Universal_newlines = false,Startupinfo = none,Creationflags = 0)

The parameter meanings are as follows:

ARGsIt must be a string or containProgramParameter List. The program to be executed is generally the first item in this list, or the string itself. But it can also be usedExecutableParameter. WhenExecutableWhen the parameter is not blank,ARGsThe first item in is still considered as the program's "command name". Unlike the real executable file name, this "command name" is a name for display, for example, if you run the ps command under * nix, the command name is displayed ".

In * nix, whenShell = false(When the program is executed, popenuses OS .exe CVP () to execute a subroutine.ARGsIt is generally a list. IfARGsIf it is a string, it will be treated as the path of the executable file, so that no parameters can be input.

Note:
Shlex. Split () can be used to serialize complex command parameters, such:

 >>  Import   shlex ,  subprocess  
   command_line =  raw_input   ( ) /bin/Vikings- input  eggs.  TXT -output  "spam spam.txt" - CMD   "Echo '$ money '"  
   ARGs =  shlex .  split   ( command_lin E )  
    Print  ARGs  [  '/bin /Vikings ',  '-input' ,  'eggs.txt' ,  '-output' ,  'spam spam.txt ', '-cmd' ,  "Echo '$ money'"  ]  
 >> P =  subprocess .  popen   ( ARGs )   # executed successfully!  

As you can see, spaces separated by spaces (such as -inputcharacter and separator (such as eggs.txt) are separated into independent items in the list, but spaces enclosed in quotation marks or escaped spaces are not listed in this column. This also has the behavior of most shells.

In * nix, whenShell = trueIfARGsIs a string, and shell is used to explain and execute this string. IfARGsIs a list, the first item is considered as a command, and the rest are considered as parameters for the shell itself. That is, it is equivalent:

 
Popen(['/Bin/Sh','-C', ArgS[0], ArgS[1],...])
 

In Windows, popen uses CreateProcess () that accepts string parameters to execute subprograms. IfARGsIs a list, it will be used firstList2domainline ()Convert to a string.

IfBufsizeThe function is the same as that of the built-in function open (). 0 indicates no buffer, 1 indicates row buffer, and other positive values indicate the number of buffer bytes. negative values indicate that the system default value is used. The default value is 0.

ExecutableParameter specifies the program to be executed. It is rarely used:ARGsParameter. IfShell = true,ExecutableIt can be used to specify the shell used for execution (such as Bash, CSH, and zsh ). * In Nix, the default value is/bin/sh. In Windows, it is the environment variable.Comspec. In Windows, only when the command you want to execute is indeed a shell built-in command (suchDir,Copy), You need to specifyShell = trueWhen you want to execute a command line-based batch processing script, you do not need to specify this option.

Stdin,StdoutAndStderrRepresents the standard input, standard output, and standard errors of subprograms respectively. The optional values include pipe (as described below), a valid file descriptor (which is actually a positive integer), a file object, and none. If pipe is used, a new pipeline needs to be created. If it is set to none, no redirection is performed. The file descriptor of the child process inherits from the parent process. In addition,StderrThe value can also be stdout (see below), indicating that the standard error of the sub-process is also output to the standard output.

IfPreexec_fnA callable object (such as a function) is called before the sub-process is executed. (* Nix only)

IfClose_fdsSet to true. * In Nix, all file descriptors except 0, 1, and 2 are disabled before the sub-process is started. In Windows, other file descriptors are not inherited.

IfShellIf it is set to true, the specified command will be interpreted and executed in shell. This is already described in detail.

IfCWDIf it is not noneCWDAs the current directory of the subroutine. Note that this directory is not used as the search object for executable files, so do not set the directory where the program files are locatedCWD.

IfEnvIf it is not none, the environment variable of the subroutine isEnvInstead of inheriting the environment variables of the parent process by default. Note: Even if you only define the value of an environment variable in env, it will prevent the subroutine from obtaining the environment variables of other parent processes (that is, if there is only one item in env, then there is only one environment variable for the sub-process ). For example:

 
>>> Subprocess.Popen('Env', ENV ={'Xxx':'123','Yyy':'Zzz'}) 
<Subprocess.Popen ObjectAt 0xb694112c> 
>>>Xxx =123YYY = zzz
 

IfUniversal_newlinesIf this parameter is set to true, stdout and stderr of the sub-process are considered as text objects, regardless of the * nix line terminator ('N'), Or the row terminator in the old Mac format ('R'), Or Windows format line terminator ('Rn') Will be considered'N'.

IfStartupinfoAndCreationflags, Will be passed toCreateProcess ()Function, used to specify other attributes of a subroutine, such as the main window style or the priority of a child process. (Windows only)

After introducing the parameters of popen, let's take a look at two things:

Subprocess.Pipe
AStdin,StdoutAndStderrThe value of the three parameters indicates that a new pipeline needs to be created.

Subprocess.Stdout
AStderrThe special input value of the parameter, indicating that the standard error of the subroutine is merged to the standard output.

Convenient Functions

Subprocess.Call(* Popenargs,** Kwargs)
Run the command and wait until the command ends before returning the return value of the sub-process. The parameter is the same as that of popen, because you will know when/usr/lib/python2.6/subprocess. py is opened, and the document is removed as follows:

DefCall(*Popenargs,**Kwargs):ReturnPopen(*Popenargs,**Kwargs).Wait()
 

Subprocess.Check_call(* Popenargs,** Kwargs)
Run the preceding call command and check the return value. If the sub-process returns a value other than 0CalledprocesserrorExceptionReturncodeAttribute to record the returned values of sub-processes.

>>> Subprocess.Check_call('False') 
Traceback(Most recent call last):
File"<Stdin>", Line1,In <Module> 
File"/Usr/lib/python2.6/subprocess. py", Line498,InCheck_callRaiseCalledprocesserror(Retcode,CMD) 
Subprocess.Calledprocesserror: Command'False'Returned non-zero exit status1
 

Exception

The exception thrown in the child process will be thrown again in the parent process. In addition, an exception is calledChild_tracebackThis is a string containing the traceback information of the sub-process error.

The most common error is oserror. For example, if an existing subroutine is executed, oserror is generated.

In addition, if the wrong parameter is used to call popen, valueerror is thrown.

When the subprogram returns a value other than 0, check_call () also producesCalledprocesserrorException.

Security

Unlike other popen functions, this module does not secretly call/bin/sh to explain the command. That is to say, every character in the command is safely transmitted to the sub-process.

Popen object

The popen object has the following methods:

Popen.Poll ()
Check whether the sub-process has ended, set and returnReturncodeAttribute.

Popen.Wait ()
Wait until the sub-process ends, set and returnReturncodeAttribute.

Note:If the sub-process outputs a large amount of data to the stdout or stderr pipeline and reaches the cache size of the system pipe, the sub-process will wait for the parent process to read the pipeline, when the parent process is wait at this time, the so-called deadlock will occur, and the consequence is very serious. RecommendedCommunicate ()To avoid this situation.

Popen.Communicate(Input=None)
Interaction with sub-processes: send data to stdin and read data from stdout and stderr until EOF is received. Wait until the child process ends. OptionalInputIf any, it must be a string type.
This function returns a tuple :(Stdoutdata,Stderrdata).
Note: to send data to the stdin of the sub-process, stdin must be Pipe When popen is used. Similarly, stdout or stderr must be pipe if data can be received.

Note: Read data will be cached in the memory, so be careful when the data volume is very large.

Popen.Send_signal(Signal)
Send to sub-processSignalSemaphores.

Note: currently, only sigterm can be sent in windows, which is equivalentTerminate ().

Popen.Terminate ()
Stop the sub-process. In POSIX, The sigterm signal is sent. In WindowsTerminateprocess ()This API.

Popen.Kill ()
Kill the child process. In POSIX, A sigkill signal is sent. Windows andTerminate ()No exception.

Popen.Stdin
IfStdinThe parameter is pipe. This attribute is a file object. OtherwiseNone.

Popen.Stdout
IfStdoutThe parameter is pipe. This attribute is a file object. OtherwiseNone.

Popen.Stderr
IfStderrThe parameter is pipe. This attribute is a file object. OtherwiseNone.

Popen.PID
The process ID of the sub-process. Note that ifShellThe parameter is true, which indicates the process Number of the sub-shell.

Popen.Returncode
The return value of a subroutine is set by poll () or wait (), and indirectly by communicate.
If it is none, the sub-process has not been terminated.
If the value is negative-N, the sub-process is terminated by the N signal. (* Nux only)

Use subprocess to replace other functions

In this section, we can use subprocess for some common examples. We assume that "from subprocess import *" is used to import modules:

Replace shell command:

Output = 'mycmd myarg'
Equivalent
Output = popen (["mycmd", "myarg"], stdout = pipe). Communicate () [0]

Replace shell pipeline:

Output = 'dmesg | grep hda'
Equivalent
P1 = popen (["dmesg"], stdout = pipe)
P2 = popen (["grep", "hda"], stdin = p1.stdout, stdout = pipe)
Output = p2.communicate () [0]

ReplaceOS. System ()

STS = OS. System ("mycmd" + "myarg ")
Equivalent
P = popen ("mycmd" + "myarg", shell = true)
STS = OS. waitpid (P. PID, 0) [1]

Note:

• Generally, you do not need to use shell to call a program.
• Subprocess can be used to obtain the return values of subprograms more conveniently.

In fact, the more real replacement is:

Try:
Retcode = call ("mycmd" + "myarg", shell = true)
If retcode <0:
Print> SYS. stderr, "Child was terminated by signal",-retcode
Else:
Print> SYS. stderr, "child returned", retcode
Counter t oserror, E:
Print> SYS. stderr, "execution failed:", E

Replacing the OS. Spawn Series
P_nowait example

PID = OS. spawnlp (OS. p_nowait, "/bin/mycmd", "mycmd", "myarg ")
Equivalent
PID = popen (["/bin/mycmd", "myarg"]). PID

P_wait example

Retcode = OS. spawnlp (OS. p_wait, "/bin/mycmd", "mycmd", "myarg ")
Equivalent
Retcode = call (["/bin/mycmd", "myarg"])

Example of Vector

OS. spawnvp (OS. p_nowait, path, argS)
Equivalent
Popen ([path] + ARGs [1:])

Examples of Environment Variables

OS. spawnlpe (OS. p_nowait, "/bin/mycmd", "mycmd", "myarg", ENV)
Equivalent
Popen (["/bin/mycmd", "myarg"], ENV = {"path": "/usr/bin "})

ReplaceOS. popen (),OS. popen2 (),OS. popen3 ():

Pipe = OS. popen ("cmd", 'R', bufsize)
Equivalent
Pipe = popen ("cmd", shell = true, bufsize = bufsize, stdout = pipe). stdout

Pipe = OS. popen ("cmd", 'w', bufsize)
Equivalent
Pipe = popen ("cmd", shell = true, bufsize = bufsize, stdin = pipe). stdin

(Child_stdin, child_stdout) = OS. popen2 ("cmd", mode, bufsize)
Equivalent
P = popen ("cmd", shell = true, bufsize = bufsize, stdin = pipe, stdout = pipe, close_fds = true)
(Child_stdin, child_stdout) = (P. stdin, P. stdout)

(Child_stdin, child_stdout, child_stderr) = OS. popen3 ("cmd", mode, bufsize)
Equivalent
P = popen ("cmd", shell = true, bufsize = bufsize, stdin = pipe, stdout = pipe, stderr = pipe, close_fds = true)
(Child_stdin, child_stdout, child_stderr) = (P. stdin, P. stdout, P. stderr)

(Child_stdin, child_stdout_and_stderr) = OS. popen4 ("cmd", mode, bufsize)
Equivalent
P = popen ("cmd", shell = true, bufsize = bufsize, stdin = pipe, stdout = pipe, stderr = stdout, close_fds = true)
(Child_stdin, child_stdout_and_stderr) = (P. stdin, P. stdout)

* In Nix, OS. popen2, OS. popen3, OS. popen4 can also accept a list as the command to execute. In this case, the parameter will be directly transmitted to the program without being interpreted and converted by shell. As follows:

(Child_stdin, child_stdout) = OS. popen2 (["/bin/ls", "-l"], mode, bufsize)
Equivalent
P = popen (["/bin/ls", "-l"], bufsize = bufsize, stdin = pipe, stdout = pipe)
(Child_stdin, child_stdout) = (P. stdin, P. stdout)

Return Value processing:

Pipe = OS. popen ("cmd", 'w ')
...
Rc = pipe. Close ()
If RC! = None and RC %256:
Print "there were some errors"
Equivalent
Process = popen ("cmd", 'w', shell = true, stdin = pipe)
...
Process. stdin. Close ()
If process. Wait ()! = 0:
Print "there were some errors"

ReplacePopen2Functions in the module:

(Child_stdout, child_stdin) = popen2.popen2 ("somestring", bufsize, Mode)
Equivalent
P = popen (["somestring"], shell = true, bufsize = bufsize, stdin = pipe, stdout = pipe, close_fds = true)
(Child_stdout, child_stdin) = (P. stdout, P. stdin)

* In Nix, popen2 can also accept a list as the command to execute. In this case, the parameter is directly transmitted to the program without being converted by Shell interpretation. As follows:

(Child_stdout, child_stdin) = popen2.popen2 (["mycmd", "myarg"], bufsize, Mode)
Equivalent
P = popen (["mycmd", "myarg"], bufsize = bufsize, stdin = pipe, stdout = pipe, close_fds = true)
(Child_stdout, child_stdin) = (P. stdout, P. stdin)

Popen2.popen3 and popen2.popen4 can also be replaced by subprocess. popen, except for the following points:

• Popen throws an exception when execution fails.
• CapturestderrParameterStderrReplace
• Stdin = PipeAndStdout = PipeMust be specified
• Popen2 disables all file descriptors by default, and popen must specifyClose_fds = true