Python多進程機制執行個體詳解

本文執行個體講述了Python多進程機制。分享給大家供大家參考。具體如下：

在以前只是接觸過PYTHON的多線程機制，今天搜了一下多進程，相關文章好像不是特別多。看了幾篇，小試了一把。程式如下，主要內容就是通過PRODUCER讀一個本地檔案，一行一行的放到隊列中去。然後會有相應的WORKER從隊列中取出這些行。

import multiprocessingimport osimport sysimport Queueimport timedef writeQ(q,obj):    q.put(obj,True,None)    print "put size: ",q.qsize()def readQ(q):    ret = q.get(True,1)    print "get size: ",q.qsize()    return retdef producer(q):    time.sleep(5)  #讓進行休息幾秒 方便ps命令看到相關內容    pid = os.getpid()    handle_file = '/home/dwapp/joe.wangh/test/multiprocess/datafile'    with open(handle_file,'r') as f:   #with...as... 這個用法今天也是第一次看到的        for line in f:            print "producer <" ,pid , "> is doing: ",line            writeQ(q,line.strip())    q.close()def worker(q):    time.sleep(5)  #讓進行休息幾秒 方便ps命令看到相關內容    pid = os.getpid()    empty_count = 0    while True:        try:            task = readQ(q)            print "worker <" , pid , "> is doing: " ,task            '''            如果這裡不休眠的話 一般情況下所有行都會被同一個子進程讀取到 為了使實驗效果更加清楚 在這裡讓每個進程讀取完一行內容時候休眠5s 這樣就可以讓其他的進程到隊列中進行讀取            '''            time.sleep(5)          except Queue.Empty:            empty_count += 1            if empty_count == 3:                print "queue is empty, quit"                q.close()                sys.exit(0)def main():    concurrence = 3    q = multiprocessing.Queue(10)    funcs = [producer , worker]    for i in range(concurrence-1):        funcs.append(worker)    for item in funcs:        print str(item)    nfuncs = range( len(funcs) )    processes = []        for i in nfuncs:        p = multiprocessing.Process(target=funcs[i] , args=(q,))        processes.append(p)    print "concurrence worker is : ",concurrence," working start"    for i in nfuncs:        processes[i].start()    for i in nfuncs:        processes[i].join()    print "all DONE"if __name__ == '__main__':    main()

實驗結果如下：

dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>python 1.py concurrence worker is : 3 working startproducer < 28320 > is doing: line 1put size: 1producer < 28320 > is doing: line 2put size: 2producer < 28320 > is doing: line 3put size: 3producer < 28320 > is doing: line 4put size: 3producer < 28320 > is doing: line 5get size: 3put size: 4worker < 28321 > is doing: line 1get size: 3worker < 28322 > is doing: line 2get size: 2worker < 28323 > is doing: line 3get size: 1worker < 28321 > is doing: line 4get size: 0worker < 28322 > is doing: line 5queue is empty, quitqueue is empty, quitqueue is empty, quitall DONE

程式運行期間在另外一個視窗進行ps命令 可以觀測到一些進程的資訊

dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28319 27481 8 14:04 pts/0  00:00:00 python 1.pydwapp  28320 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28325 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 python     #此時28320進程 也就是PRODUCER進程已經結束dwapp  28319 27481 1 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28328 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28319 27481 0 14:04 pts/0  00:00:00 python 1.pydwapp  28321 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28322 28319 0 14:04 pts/0  00:00:00 python 1.pydwapp  28323 28319 0 14:04 pts/0  00:00:00 [python]   #這裡應該是代表28323進程（WORKER）已經運行結束了dwapp  28331 27849 0 14:04 pts/13  00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp  13735 11830 0 Nov20 pts/12  00:00:05 pythondwapp  28337 27849 0 14:05 pts/13  00:00:00 grep python

希望本文所述對大家的Python程式設計有所協助。

