Python multi-process mechanism example detailed

This example describes the Python multi-process mechanism. Share to everyone for your reference. as follows:

In the past has only been exposed to Python multithreading mechanism, today searched a lot of progress, the relevant article seems not particularly much. After reading a few, a small try. Program as follows, the main content is to read a local file through the producer, a line of line to put in the queue. Then there will be a corresponding worker to remove the rows from the queue.

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 This is the same as the same as the same as the same as the same as the same as the other, at the same ().

import multiprocessing import OS import sys import Queue import time Def writeq (q, OBJ): q.put (obj,true,none) print "Put size:", Q.qsize () def READQ (q): ret = Q.get (true,1) print "Get Size:", q.qsize () ret URN ret def producer (q): Time.sleep (5) #让进行休息几秒 convenient PS command See related content pid = Os.getpid () handle_file = '/home/dwapp/joe.wangh/test/m Ultiprocess/datafile ' with open (Handle_file, ' R ') as F: #with ... as ... This usage is also seen for the first time today for line in F:print "producer <", PID, "> Are doing:", line Writeq (Q,line.strip ()) q.close () def wor Ker (q): Time.sleep (5) #让进行休息几秒 convenient PS command to see related content pid = Os.getpid () empty_count = 0 while True:try:task = READQ (q) print "Work Er < ", PID, > is doing:", task "' If it's not dormant, then all rows will be read by the same child process toTo make the experiment more clear here let each process read a line of content at the time of hibernation 5s so that other processes can be read in the queue ' Time.sleep (5) except Queue.Empty:empty_count + + 1 if Empty_coun t = = 3:print "Queue is empty, quit" Q.close () sys.exit (0) def main (): concurrence = 3 Q = multiprocessing. Queue (Funcs = [producer, worker] for I in range (concurrence-1): Funcs.append (worker) for item in Funcs:print STR (ite m) Nfuncs = Range (len (funcs)) processes = [] for i in nfuncs:p = multiprocessing. Process (Target=funcs[i], args= (q,)) Processes.append (p) Print "Concurrence worker are:", concurrence, "working start" for I in Nfuncs:processes[i].start () for I-nfuncs:processes[i].join () print "All done" if __name__ = = ' __main__ ': Main ()

The experimental results are as follows:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 The

The PS command in another window during program operation can observe the information of some processes

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef | gr EP Python Dwapp 13735 11830 0 Nov20 pts/12 00:00:05 python dwapp 28319 27481 8 14:04 pts/0 00:00:00 python 1.py dwapp 2832 0 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28321 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28323 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28325 27849 0 14:04 PTS/13 00:00:00 grep Python Dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef | grep python dwapp 13735 11830 0 Nov20 pts/12 00:00:05 python #此时28320进程 that is, the producer process has ended Dwapp 28319 27481 1 14:04 pts/0 00:00:00 python 1.py dwapp 28321 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.py Dwapp 28323 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28328 27849 0 14:04 pts/13 00:00:00 grep python dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef | grep python dwapp 13735 11830 0 Nov20 pts/12 00:00:05 python dwapp 28319 27481 0 14:04 pts/0 00:00:00 python 1.py dwapp 28 321 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.py dwapp 28323 28319 0 14:0 4 pts/0 00:00:00 [python] <defunct> #这里应该是代表28323进程 (WORKER) has run over Dwapp 28331 27849 0 14:04 pts/13 00:00:00 grep py Thon Dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps-ef | grep python dwapp 13735 11830 0 Nov20 pts/12 00:00:05 python dwapp 28337 27849 0 14:05 pts/13 00:00:00 grep python

I hope this article will help you with your Python programming.