Use PIPE in Python to operate Linux pipelines and pythonpipe

Use PIPE in Python to operate Linux pipelines and pythonpipe

In Linux, process communication methods include signal, pipeline, shared memory, and Message Queue socket. The pipeline is the oldest form of communication between * nix system processes. All * nix provide this communication mode. A pipe is a half-duplex communication mechanism, that is, it can only be used to read at one end and write at the other end. In addition, the pipe can only be used to communicate between two processes with common ancestor. Pipeline communication follows the principle of first-in-first-out, and data can only be read once. When this segment of data is read, it will immediately disappear from the data. This is very important.

On Linux, the pipe function is used to create an MPS queue. After the pipe function is executed, two file descriptors are generated, read and write. Pipelines in a single process have almost no function. Generally, pipe is called first and fork is called to create an IPC channel from the parent process to the child process.

In Linux, pipelines are often used. For example, when you use the cat command to view a large file, only one page cannot be displayed. You can use cat xxx | more to display it by page, for example, you can use cat xxx | grep search to search the content in a file. Here we use pipelines. Next, I will write a program for automatic paging display in python, instead of using pipelines manually.

#!/usr/bin/env pythonimport os,sysif not sys.argv[1:]:  print "No filename input"  sys.exit(1)try:    fp = open(sys.argv[1],"r")except IOError,msg:  sys.exit(msg)pi=os.pipe()pid=os.fork()if pid:  #parent  os.close(pi[0]) #close read pipe  #write to pipe  line=fp.readline()  while line:    os.write(pi[1],line)    line=fp.readline()  #close write pipe  os.close(pi[1])  #wait for chile  os.waitpid(pid,0)else:  os.close(pi[1]) #close write pipe  #put pipe read to stdin  os.dup2(pi[0],sys.stdin.fileno())  os.close(pi[0])  os.execl("/bin/more","more")

Save this code as scat. py to run scat. py file name, the system will automatically read the file content and pagination, the same effect as using cat file name | more. In the above Code, fork, dup2, and exec functions described in previous blogs are used.

First, the program creates a pipeline. After the system fork, the parent process closes its read end, the child process closes its write end, and the parent process reads the passed file name, write the content to the MPs queue through the MPs queue's write port, close the write port, and wait until the sub-process ends. After the sub-process closes the write port, It redirects the read port to the standard input of the process, and the sub-process can automatically receive the data transmitted from the pipeline, finally, the execl function is used to call the more program of the system to process the passed data, so that the paging effect can be easily achieved.

Pipe is a half-duplex communication mechanism. If full-duplex communication is required between processes, you can create two pipelines to achieve full-duplex communication. In addition, the pipe anonymous pipeline can only be used for communication between processes with the same parent process. * nix provides another fifo (named pipeline) to allow communication between any process, in the following blog.