Subprocess Learning in Python

Life is endless struggle!

The purpose of subprocess is to start a new process and communicate with it.

Only one class is defined in the Subprocess module: Popen. You can use Popen to create processes and make complex interactions with processes. Its constructor is as follows:

Subprocess. Popen (args, bufsize=0, Executable=none, Stdin=none, Stdout=none, Stderr=none, Preexec_fn=none, Close_fds=false, shell= False, Cwd=none, Env=none, Universal_newlines=false, Startupinfo=none, creationflags=0)

The parameter args can be a string or sequence type (for example: list, tuple) that specifies the process's executable file and its arguments. If it is a sequence type, the first element is usually the path to the executable file. We can also explicitly use the executeable parameter to specify the path to the executable file.

The parameters stdin, stdout, stderr respectively represent the standard input, output, and error handles of the program. They can be pipe, file descriptor or file object, or set to none, which means inheriting from the parent process.

If the parameter shell is set to True, the program executes through the shell.

The parameter env is a dictionary type that specifies the environment variables for the child process. If env = None, the environment variables of the child process are inherited from the parent process.

Subprocess. PIPE

When you create a Popen object, subprocess. Pipe can initialize stdin, stdout, or stderr parameters. Represents a standard stream that communicates with a child process.

Subprocess. STDOUT

When you create a Popen object, the stderr parameter is initialized to indicate that the error is output through the standard output stream.

Methods of Popen:

Popen.poll ()

Used to check whether the child process has ended. Sets and returns the ReturnCode property.

Popen.wait ()

Wait for the child process to end. Sets and returns the ReturnCode property.

Popen.communicate (Input=none)

Interacts with the child process. Send data to stdin, or read data from stdout and stderr. Optional parameter input Specifies the parameters that are sent to the child process. Communicate () returns a tuple: (Stdoutdata, Stderrdata). Note: If you want to send data to it through the process's stdin, the parameter stdin must be set to pipe when the Popen object is created. Similarly, if you want to get data from stdout and stderr, you must set stdout and stderr to pipe.

Popen.send_signal (signal)

Sends a signal to a child process.

Popen.terminate ()

Stop (stop) child process. Under the Windows platform, this method will call Windows API TerminateProcess () to end the child process.

Popen.kill ()

Kills a child process.

Popen.stdin,popen.stdout, Popen.stderr, the official document says:

stdin, stdout and stderr Specify the executed programs ' standard input, standard output and STA Ndard error file handles, respectively. Valid values is PIPE, an existing file descriptor (a positive integer), an existing file object, and none.

Popen.pid

Gets the process ID of the child process.

Popen.returncode

Gets the return value of the process. Returns none if the process is not finished yet.

---------------------------------------------------------------

Simple to use:

P=subprocess. Popen ("dir", Shell=true)  

The shell parameter is determined according to the command you want to execute, which is the dir command, it must be Shell=true, p.wait () can get the return value of the command.

If it is written as a=p.wait (), A is returncode. If you output a, it is possible that 0 "indicates successful execution."

---------------------------------------------------------------------------

Process Communication

If you want the output of a process, the pipeline is a convenient way to:

P=subprocess. Popen ("dir", Shell=true, stdout=subprocess. PIPE, Stderr=subprocess. PIPE)  (stdoutput,erroutput) = P.<span>commu</span>nicate ()  

P.communicate will wait until the process exits and return the standard output and standard error output so that the child process output can be obtained.

Let's look at a communicate example.

The above example sends data to STDIN via communicate and then uses a tuple to receive the execution result of the command.

------------------------------------------------------------------------

Above, the standard output and the standard error output are separate, or can be combined, simply set the stderr parameter to subprocess. StdOut on it, this way:

P=subprocess. Popen ("dir", Shell=true, stdout=subprocess. PIPE, Stderr=subprocess. STDOUT)  (stdoutput,erroutput) = P.<span>commu</span>nicate ()

If you want to handle the output of a child process in a row, there is no problem:

P=subprocess. Popen ("dir", Shell=true, stdout=subprocess. PIPE, Stderr=subprocess. STDOUT) while  True:      buff = p.stdout.readline ()      if buff = = "and P.poll ()! = None:          break  

Dead lock

But if you use a pipe and don't handle the output of the pipeline, be careful, if the child process outputs too much data, a deadlock can occur, such as the following usage:

P=subprocess. Popen ("Longprint", Shell=true, stdout=subprocess. PIPE, Stderr=subprocess. STDOUT)  

Longprint is an imaginary process with a lot of output, so in my XP, Python2.5 environment, when the output reaches 4096, the deadlock occurs. Of course, if we use P.stdout.readline or p.communicate to clean up the output, then no matter how much output, the deadlock will not happen. Or we can avoid deadlocks by not using pipelines, such as not redirecting, or redirecting to files.

----------------------------------

Subprocess can also be connected to multiple commands to execute.

In the shell we know that you want to connect multiple commands to use a pipeline.

In subprocess, the output from the previous command can be used as input for the next execution. Examples are as follows:

Example, P2 uses the result of the first execution of the command P1 stdout as the input data, and then executes the tail command.

- -------------------

The following is a larger example. Used to ping a series of IP addresses and output whether the hosts of these addresses are alive. The code references the Python UNIX Linux System Administration Guide.

#!/usr/bin/env python from    threading import Thread  import subprocess from  queue import queue    Num_ threads=3  ips=[' 127.0.0.1 ', ' 116.56.148.187 ']  q=queue ()  def Pingme (i,queue): While      True:          ip= Queue.get ()          print ' Thread%s pinging%s '% (I,IP)          ret=subprocess.call (' ping-c 1%s '% Ip,shell=true,stdout=open ('/dev/null ', ' W '), stderr=subprocess. STDOUT)          if ret==0:              print '%s is alive! '%ip          elif ret==1: "              print '%s is down ... '%ip          queue.task_ Done ()    #start num_threads threads for  i in range (num_threads):      t=thread (target=pingme,args= (I,Q))      T.setdaemon (True)      T.start () for    IP in IPs:      q.put (IP)  print ' main thread waiting ... '  

The main benefit of using subprocess in the code above is that using multiple threads to ping commands can save a lot of time.

Assuming that we use a thread to process, each ping waits until the previous one finishes and pings the other address. So if there are 100 addresses, the total time required is =100* average time.

If multiple threads are used, the maximum execution time is the total elapsed time of the entire program. "Time saves more than a single thread"

Pay attention to the learning of the queue module here.

The execution of the PINGME function is this:

The thread that starts is going to execute the PINGME function.

The PINGME function detects if there are elements in the queue. If so, remove and execute the ping command.

This queue is shared by multiple threads. So we don't use lists here. "Assuming we use the list here, we need to do the synchronization control ourselves." The queue itself has been synchronously controlled by the semaphore, saving ourselves from doing synchronous control work =. =

The join function of Q in the code is blocking the current thread. The following is an e-text comment

　Queue. Join ()

Blocks until all items in the queue has been gotten and processed (Task_done ()).

---------------------------------------------

When I learned the processing module, I encountered the join function of the process. The join function of the process means that the wait process is finished running.

Subprocess Learning in Python