. Python multi-process manager process Pool

Powerful Manager Module

The previous section implements data sharing in a way that has only two structures, value and array.

Python provides a powerful manager module dedicated to data sharing.

He supports a number of types, including:Value, Araay, List, dict, Queue, Lock , and so on.

The following example:

Import multiprocessingdef Worker (D,l): L + = range (one, all) for I in xrange (1, 6): Key = "key{0}". Format (i) val = "val{0}". Format (i) D[key] = valif __name__ = = "__main__": Manager = multiprocessing. Manager () d = manager.dict () L = manager.list () p = multiprocessing. Process (Target=worker, args= (d, L)) P.start () P.join () print (d) print (L)

Printing results:

{' Key3 ': ' Val3 ', ' key2 ': ' Val2 ', ' key1 ': ' Val1 ', ' key5 ': ' val5 ', ' key4 ': ' Val4 '} [11, 12, 13, 14, 15]

Process Pool:

The pool can provide a specified number of processes for the user to call, and when a new request is submitted to the pool,

If the pool is not full, a new process is created to execute the request;

However, if the number of processes in the pool has reached the specified maximum, then the request waits until the process ends in the pool before a new process is created.

the difference between blocking and non-blocking :

Pool.apply_async non-blocking, the maximum number of process pool processes defined can be executed concurrently.

Pool.apply a process is finished and released back to the process pool before the next process can begin

Example:

Non-blocking:

Import Multiprocessingimport timedef Worker (msg): Print ("###### #start {0}########". Format (msg)) Time.sleep (1) PR Int ("###### #end {0}########". Format (msg)) if __name__ = = "__main__": Pool = multiprocessing. Pool (processes=3) for I in xrange (1, ten): Msg = "hello{0}". Format (i) Pool.apply_async (Func=worker, args= ( MSG,)) Pool.close () Pool.join () #调用join之前, call the close function first, or else an error occurs. No new processes are added to the Pool,join function to wait for all child processes to end print ("main end") after close execution

Printing results:

###### #start hello1############## #start hello2############## #start hello3############## #end hello1############### Start hello4############## #end hello2############## #start hello5############## #end hello3############## #start hello6############## #end hello4############## #start hello7############## #end hello5############## #start hello8#### ########## #end hello6############## #start hello9############## #end hello7############## #end hello8############### End hello9####### #main End

Blocking:

Import Multiprocessingimport timedef Worker (msg): Print ("###### #start {0}########". Format (msg)) Time.sleep (1) PR Int ("###### #end {0}########". Format (msg)) if __name__ = = "__main__": Pool = multiprocessing.     Pool (processes=3) for I in xrange (1,): Msg = "hello{0}". Format (i) pool.apply (Func=worker, args= (msg,)) Pool.close () Pool.join () #调用join之前, call the close function first, or you will get an error. No new processes are added to the Pool,join function to wait for all child processes to end print ("main end") after close execution

Printing results:

###### #start hello1############## #end hello1############## #start hello2############## #end hello2############### Start hello3############## #end hello3############## #start hello4############## #end hello4############## #start hello5############## #end hello5############## #start hello6############## #end hello6############## #start hello7#### ########## #end hello7############## #start hello8############## #end hello8############## #start hello9############## #end hello9####### #main End

Compare the two types of output states to understand.

. Python multi-process manager process Pool