Python's multiprocessing multi-process communication pipe and queue Introduction _php tutorial

Description of the pipe and queue for Python's multiprocessing multi-process communication

Python's multiprocessing provides IPC (pipe and queue), making Python multi-process concurrency more efficient. In this article we will introduce the pipe and queue in detail.

These two days warm so Python's multiprocessing multi-process module, see Pipe and queue these two IPC way, what matter IPC? IPC is the inter-process communication mode, half of which is socke,rpc,pipe and message queue.


The pipe and the queue will be engaged again today.

The code is as follows

#coding: Utf-8 Import multiprocessing Import time def proc1 (pipe): While True: For I in Xrange (10000): Print "Send%s"%i Pipe.send (i) Time.sleep (1) def proc2 (pipe): While True: print ' PROC2 receive: ', PIPE.RECV () Time.sleep (1) def proc3 (pipe): While True: print ' PROC3 receive: ', PIPE.RECV () Time.sleep (1) # Build A pipe Pipe = multiprocessing. Pipe () Print pipe # Pass an end of the pipe to process 1 P1 = multiprocessing. Process (Target=proc1, args= (Pipe[0],)) # Pass The other end of the pipe to process 2 P2 = multiprocessing. Process (TARGET=PROC2, args= (Pipe[1],)) P1.start () P2.start () P1.join () P2.join ()

Not only multiprocessing pipe, including other pipe implementations, are just two sessions between the play, I give you, you come to receive or you come, I receive. Of course, you can also make a duplex state.

Queue, you can have more processes to participate in. Usage is about the same as some other queue.


See the Xia Guan Web document:

Multiprocessing. Pipe ([duplex])

Returns a pair (conn1, conn2) of Connection objects representing the ends of a pipe.

#两个pipe对象. Use these two objects to communicate with each other.


If Duplex is True (the default) then the pipe is bidirectional. If Duplex is False then the pipe was unidirectional:conn1 can only being used for receiving messages and CONN2 can only be us Ed for sending messages.


Class multiprocessing. Queue ([maxsize])

Returns a process Shared queue implemented using a pipe and a few locks/semaphores. When a process first puts an item on the queue a feeder thread was started which transfers objects from a buffer into the P Ipe.

#队列的最大数


The usual queue.empty and Queue.full exceptions from the standard library ' s Queue module is raised to signal timeouts.


Queue implements all the methods of Queue.queue except for Task_done () and join ().


Qsize ()

Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable.

#队列的大小


Note that this could raise notimplementederror on the Unix platforms like Mac OS X where Sem_getvalue () are not implemented.


Empty ()

Return True If the queue is empty, False otherwise. Because of Multithreading/multiprocessing semantics, this is not reliable.

#是否孔了. If it is empty, he returns a status of true.


Full ()

Return True If the queue is full, False otherwise. Because of Multithreading/multiprocessing semantics, this is not reliable.

#队列的状态是否满了.


Put (obj[, block[, timeout])

Put obj into the queue. If the optional argument block is True (the default) and timeout are None (the default), block if necessary until a free SL OT is available. If timeout is a positive number, it blocks at most timeout seconds and raises the queue.full exception if no free slots was Available within that time. Otherwise (Block is False), put an item on the queue if a free slot is immediately available, else raise the Queue.full ex Ception (timeout is ignored in).

#塞入队列, you can add a time-out.

Put_nowait (obj)

Equivalent to put (obj, False).

#这里是不堵塞的


Get ([block[, timeout]])

Remove and return an item from the queue. If optional args block is True (the default) and timeout are None (the default), block if necessary until an item is Availa ble. If timeout is a positive number, it blocks at most timeout seconds and raises the Queue.empty exception if no item was AVA Ilable within that time. Otherwise (Block is False), return a item if one is immediately available, else raise the Queue.empty exception (timeout is ignored in the case).

#获取状态


Get_nowait ()

Equivalent to get (False).

#不堵塞的get队列里面的数据


Queue has a few additional methods not found in Queue.queue. These methods is usually unnecessary for most code:


Close ()

Indicate that's no more data would be put in this queue by the current process. The background thread would quit once it had flushed all buffered data to the pipe. This was called automatically when the queue was garbage collected.

#关闭, save resources for the current process.



I configured the multiprocessing team length is 3, and then when I put in the fourth one, I will find a blockage, he is waiting, someone put the data get off one, then he can continue to plug in. If you use Put_nowait (), the queue will be an error immediately.


/system/library/frameworks/python.framework/versions/2.7/lib/python2.7/multiprocessing/queues.pyc in Put_nowait ( Self, obj)


/system/library/frameworks/python.framework/versions/2.7/lib/python2.7/multiprocessing/queues.pyc in put (self, OBJ, block, timeout)




Here is a test of the code, students can run the demo, feel the next.

The code is as follows

#coding: Utf-8 Import OS Import multiprocessing Import time # Write Worker def inputq (queue): While True: info = "Process number%s: Time:%s"% (Os.getpid (), int (Time.time ())) Queue.put (Info) Time.sleep (1) # Get Worker def outputq (Queue,lock): While True: info = Queue.get () # Lock.acquire () Print (str (os.getpid ()) + ' (GET): ' + info) # lock.release () Time.sleep (1) #=================== # Main Record1 = [] # Store input processes Record2 = [] # Store output processes Lock = multiprocessing. Lock () # to prevent messy print Queue = multiprocessing. Queue (3) # Input Processes For I in range (10): Process = multiprocessing. Process (target=inputq,args= (queue,)) Process.Start () Record1.append (Process) # Output processes For I in range (10): Process = multiprocessing. Process (target=outputq,args= (Queue,lock)) Process.Start () Record2.append (Process)

Well, the use of pipe and queue is a simple story. In fact, I would like to pull a python pipe, the result of Google a search, saw the pipe multiprocessing. After I finished the pipe, I felt that the content of the article was too small, so I added the queue ...

http://www.bkjia.com/PHPjc/968077.html www.bkjia.com true http://www.bkjia.com/PHPjc/968077.html techarticle Python's multiprocessing multi-process communication pipe and queue Introduction Python's multiprocessing provides IPC (pipe and queue), making Python multi-process concurrency more efficient. In this article we will come in detail ...

