Eight weeks five sessions (December 15)
16.1 multi-process Message Queuing
16.2 Message Queuing pipe
1. Message Queuing
Message Queuing is the container that holds messages during the transmission of messages.
The most classic usage of Message Queuing is that consumers and generators pass messages through message pipelines, and consumers and generators are not the process. The producer writes a message to the pipeline, and the consumer reads the message from the pipeline.
The operating system provides a number of mechanisms for inter-process communication, and the multiprocessing module provides two ways to implement Queue and Pipe.
The message queue is implemented through the Pipe inside the mutiprocess:
Example 1:
from multiprocessing import Queue, Process def write(q): ##向队列里循环写入列表的数据 for i in [‘a‘, ‘b‘, ‘c‘, ‘d‘]: q.put(i) print ‘put {0} to queue.‘.format(i)def read(q): ##循环去队列里面的数据 while 1: result = q.get() print ‘get {0} from queue.‘.format(result)def main(): q = Queue() #创建队列对象 pw = Process(target=write, args=(q,)) #创建写进程 pr = Process(target=read, args=(q,)) #创建读进程 pw.start() pr.start() pw.join() pr.terminate() # pr进程里是死循环,无法等待其结束,只能强行终止;terminate方法实现if __name__ == ‘__main__‘: main()#结果:put a to queue.put b to queue.put c to queue.put d to queue.
2. Message Queuing PIPE
- The Pipe method return (CONN1,CONN2) represents the two end of a pipeline. The pipe method has the duplex parameter, if the duplex parameter is True (the default), then the pipeline is full duplex, that is, both CONN1 and CONN2 can send and receive. Duplex is only responsible for receiving messages for FALSE,CONN1, CONN2 is only responsible for sending messages.
- The Send and Recv methods are methods for sending and receiving messages, respectively. The Close method means that the pipeline is closed and the pipe is closed when the message is accepted.
Example 2:
from multiprocessing import Pipe, Processimport timedef proc1(pipe): for i in xrange(1, 10): pipe.send(i) print ‘send {0} to pipe‘.format(i) time.sleep(1)def proc2(pipe): n = 9 while n > 0: result = pipe.recv() print ‘recv {0} from pipe‘.format(result) time.sleep(1) n -= 1def main(): pipe = Pipe(duplex=False) #一边负责接收,一边只负责发送 print type(pipe) p1 = Process(target=proc1, args=(pipe[1],)) p2 = Process(target=proc2, args=(pipe[0],)) p1.start() p2.start() p1.join() p2.join() pipe[0].close() pipe[1].close()if __name__ == ‘__main__‘: main()#结果:<type ‘tuple‘>send 1 to piperecv 1 from pipesend 2 to piperecv 2 from pipesend 3 to piperecv 3 from pipesend 4 to piperecv 4 from pipesend 5 to piperecv 5 from pipesend 6 to piperecv 6 from pipesend 7 to piperecv 7 from pipesend 8 to piperecv 8 from pipesend 9 to piperecv 9 from pipe
Python learning-eight weeks five lessons (December 15)