Python subprocess. Popen communicate () and wait () usage difference, pythonpopenwait

Python subprocess. Popen communicate () and wait () usage difference, pythonpopenwait

The reason for this problem is that the program has not returned since the wait method of Popen is called. Google found that wait may cause deadlocks. To thoroughly solve this problem, search for some information to see:

Link: http://blog.csdn.net/carolzhang8406/article/details/22286913

See other people's example:

A problem encountered today. Simply put, usesubprocessModulePopenCall an external program. If stdoutOrstderrWhen the parameter is pipe and the program output exceeds the pipe size of the operating system Popen.wait()Wait until the end of the program to obtain the returned value, resulting in a deadlock, the program is stuck inwait()Call.

ulimit -aThe pipe size is 4 kb, which is only the size of each page. The query shows that the default pipe size in linux is 64 KB.

Example:

#!/usr/bin/env python# coding: utf-8# yc@2013/04/28import subprocessdef test(size):    print 'start'    cmd = 'dd if=/dev/urandom bs=1 count=%d 2>/dev/null' % size    p = subprocess.Popen(args=cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, close_fds=True)    #p.communicate()    p.wait()    print 'end'# 64KBtest(64 * 1024)# 64KB + 1Btest(64 * 1024 + 1)

First, test the output size of 64KB. Dd is used to generate a standard output of exactly 64 KB.subprocess.PopenAnd then usewait()WaitddThe call ends. You can see the correctstartAndendOutput, and then test more than 64 KB. In this case, only outputstartThat is to say, the program execution is stuck inp.wait()Program deadlock. The specific output is as follows:

startendstart

How can we avoid the deadlock problem? Recommended in official documentsPopen.communicate(). This method will put the output in the memory instead of in the pipeline, so the upper limit is related to the memory size at this time, generally there will be no problem. If you want to obtain the program return value, you can callPopen.communicate()Then retrievePopen.returncode.

Conclusion: If you usesubprocess.Popen, Do not usePopen.wait()And use Popen.communicate()Wait until the execution of the external program ends.

Popen.Wait()Bytes ¶

Wait for child process to terminate. Set and returnReturncodeAttribute.

Warning

This will deadlock when usingStdout = PIPEAnd/orStderr = PIPEAnd the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data. UseCommunicate ()To avoid that.

Popen. Communicate ( Input = None )Bytes ¶

Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate. The optionalInputArgument shoshould be a string to be sent to the child process, orNone, If no data shoshould be sent to the child.

Communicate ()Returns a tuple(Stdoutdata, stderrdata).

Note that if you want to send data to the process's stdin, you need to create the Popen objectStdin = PIPE. Similarly, to get anything otherNoneIn the result tuple, you need to giveStdout = PIPEAnd/orStderr = PIPEToo.

Note

The data read is buffered in memory, so do not use this method if the data size is large or unlimited.

Two subprocess methods:

1) If you want to directly block the call to the end of the subroutine call:

Depending on how you want to work your script you have two options. If you want the commands to block and not do anything while it is executing, you can just usesubprocess.call.

# Start and block until donesubprocess. call ([data ["om_points"], ">", diz ['D'] + "/points. xml"])


2) non-blocking mode:

If you want to do things while it is executing or feed thingsstdin, You can usecommunicateAfterpopenCall.

#start and process things, then waitp = subprocess.Popen(([data["om_points"], ">", diz['d']+"/points.xml"])print "Happens while running"p.communicate() #now wait

As stated in the documentation,waitCan deadlock, so communicate is advisable.