python多進程

標籤：

開獎

 

1、Process類

from multiprocessing import Processdef func(name):    print(‘hello‘, name)if __name__ == ‘__main__‘:    p = Process(target=func, args=(‘bob‘,))    p.start()    p.join()

 

 

2、上下文和開始方法

開始方法

import multiprocessing as mpdef foo(q):    q.put(‘hello‘)if __name__ == ‘__main__‘:    mp.set_start_method(‘spawn‘)    q = mp.Queue()    p = mp.Process(target=foo, args=(q,))    p.start()    print(q.get())    p.join()

 

上下文

import multiprocessing as mpdef foo(q):    q.put(‘hello‘)if __name__ == ‘__main__‘:    ctx = mp.get_context(‘spawn‘)    q = ctx.Queue()    p = ctx.Process(target=foo, args=(q,))    p.start()    print(q.get())    p.join()

 

 

3、進程間交換對象

Queues

from multiprocessing import Process, Queuedef func(q):    q.put([42, None, ‘hello‘])if __name__ == ‘__main__‘:    q = Queue()    p = Process(target=func, args=(q,))    p.start()    print(q.get())    # prints "[42, None, ‘hello‘]"    p.join()

 

Pipes

from multiprocessing import Process, Pipedef func(conn):    conn.send([42, None, ‘hello‘])    conn.close()if __name__ == ‘__main__‘:    parent_conn, child_conn = Pipe()    p = Process(target=func, args=(child_conn,))    p.start()    print(parent_conn.recv())   # prints "[42, None, ‘hello‘]"    p.join()

 

 

4、進程間同步

例如你可以使用一個鎖來確保只有一個進程列印到標準輸出

from multiprocessing import Process, Lockdef func(lock, i):    lock.acquire()    try:        print(‘hello world‘, i)    finally:        lock.release()if __name__ == ‘__main__‘:    lock = Lock()    for num in range(10):        Process(target=func, args=(lock, num)).start()

 

 

5、進程間共用狀態

Shared memory

使用 Value 或 Array，資料可以儲存在一個共用記憶體映射 。例如，下面的代碼

from multiprocessing import Process, Value, Arraydef f(n, a):    n.value = 3.1415927    for i in range(len(a)):        a[i] = -a[i]if __name__ == ‘__main__‘:    num = Value(‘d‘, 0.0)    arr = Array(‘i‘, range(10))    p = Process(target=f, args=(num, arr))    p.start()    p.join()    print(num.value)    print(arr[:])

參數‘ d ‘表示雙精確度浮點數，參數“i”表示一個帶正負號的整數。這些共用對象將進程和安全執行緒的。

 

Server process

通過 Manager() ，返回一個管理器對象，用以控制持有Python對象的服務進程，並允許其他進程使用代理來操作它們。

通過 Manager() 返回的管理器對象支援的類型有：list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array.

from multiprocessing import Process, Managerdef f(d, l):    d[1] = ‘1‘    d[‘2‘] = 2    d[0.25] = None    l.reverse()if __name__ == ‘__main__‘:    with Manager() as manager:        d = manager.dict()        l = manager.list(range(10))        p = Process(target=f, args=(d, l))        p.start()        p.join()        print(d)        print(l)

服務進程管理器比使用共用記憶體對象更加靈活，因為他們可以支援任意的物件類型。同樣，一個管理器可以在不同的電腦通過網際網路共用的過程。然而，他們慢於使用共用記憶體。

 

 

6、使用進程池

from multiprocessing import Poolfrom time import sleepdef f(x):    return x*xif __name__ == ‘__main__‘:    # start 4 worker processes    with Pool(processes=4) as pool:        # print "[0, 1, 4,..., 81]"        print(pool.map(f, range(10)))        # print same numbers in arbitrary order        for i in pool.imap_unordered(f, range(10)):            print(i)        # evaluate "f(10)" asynchronously        res = pool.apply_async(f, [10])        print(res.get(timeout=1))             # prints "100"        # make worker sleep for 10 secs        res = pool.apply_async(sleep, [10])        print(res.get(timeout=1))             # raises multiprocessing.TimeoutError    # exiting the ‘with‘-block has stopped the pool

 

python多進程