Python subprocess sub-process and pipeline Interaction

A long time ago, I wrote a series of interactions between Python and C ++, as shown below:

Http://blog.csdn.net/marising/archive/2008/08/28/2845339.aspx

The purpose is to solve the interoperability problem between Python and C/C ++. If the performance bottleneck is written in C, and some peripheral work is completed in Python, it is not a perfect combination.

Today, we found that a more convenient way is to use the subprocess module to create sub-processes and then use pipelines for interaction. This method is very common in shell, such as cat xxx. file | test. py is the pipeline used. In addition, in hadoop, the stream mode is the pipeline used.

In python, there are many ways to interact with shell scripts and other programs, such:

OS. System (CMD) and OS. system only execute a shell command.

OS. Open (CMD) can interact with each other, but it is a one-time process. The number of processes created and destroyed in a few calls results in poor performance.

 

Therefore, subprocess is recommended, but subprocess is more complex. You can refer to Python docs:

Http://docs.python.org/library/subprocess.html

 

Let's take a look at a simple example and call the LS command. There is no interaction between the two:

Import subprocess <br/> P = subprocess. popen ('LS ') 

Let's look at the example of getting output in the program:

Import subprocess <br/> P = subprocess. popen ('Ls', stdout = subprocess. Pipe) <br/> Print P. stdout. readlines () 

Let's take a look at an example of input and output. The parent process sends 'say Hi', the child process outputs test say hi, and the parent process retrieves and prints the output.

 

# Test1.py <br/> Import sys <br/> line = sys. stdin. readline () <br/> Print 'test', line <br/> # Run. PY <br/> from subprocess import * <br/> P = popen ('. /test1.py ', stdin = pipe, stdout = pipe) <br/> P. stdin. write ('Say hi/N') <br/> Print p. stdout. readline () <br/> # result <br/> test say hi 

Let's look at the example of continuous input and output.

Test. py

Import sys <br/> while true: <br/> line = sys. stdin. readline () <br/> if not line: Break <br/> sys. stdout. write (line) <br/> sys. stdout. flush () 

Run. py

Import sys <br/> from subprocess import * <br/> proc = popen ('. /test. py', stdin = pipe, stdout = pipe, shell = true) <br/> for line in SYS. stdin: <br/> Proc. stdin. write (line) <br/> Proc. stdin. flush () <br/> output = Proc. stdout. readline () <br/> sys. stdout. write (output) 

Note: Remember to clear the buffer in flush of run. py and flush in test. py; otherwise, the program will not get the correct input and output.

 

C/C ++ is similar. The pseudocode is as follows:

Char * line = new char [2048]; <br/> while (fgets (line, 2028, stdin) {<br/> printf (line ); <br/> fflush (stdout); // The buffer must be cleared. <br/>} 

 

For more information about popen parameter settings, see Python docs.