Python multi-process concurrency (multiprocessing) Usage examples

The examples in this article describe Python multiprogramming (multiprocessing) usage. Share to everyone for your reference. The specific analysis is as follows:

Because of the limitations of Python design (I'm talking about our usual cpython). Up to 1 CPU cores can be used.

Python provides a very useful multiprocessing package, and you just need to define a function that Python will do everything else for you. With this package, you can easily complete transitions from single process to concurrent execution.

1. New single process

If we create a small number of processes, we can do the following:

?

1 2 3 4 5 6 7 8 9 10 11

Import multiprocessing Import Time def func (msg): To I in Xrange (3): Print msg time.sleep (1) If __name__ = "__main__": P = multiprocessing. Process (Target=func, args= ("Hello",)) P.start () P.join () print "sub-process done."

2. Use Process Pool

Yes, you're not mistaken, not a thread pool. It allows you to run a full multi-core CPU, and the method is very easy to use.

Notice to use Apply_async, if drop async, become blocked version.

Processes=4 is the maximum number of concurrent processes.

?

1 2 3 4 5 6 7 8 9 10 11 12 13-14

Import multiprocessing Import Time def func (msg): To I in Xrange (3): Print msg time.sleep (1) If __name__ = "__main__": P Ool = multiprocessing. Pool (processes=4) for I in Xrange (a): msg = "Hello%d"% (i) Pool.apply_async (func, (msg,)) Pool.close () Pool.join () Prin T "sub-process (es) done."

3, use pool, and need to pay attention to the results

More often than not, we need to do more than just process execution, and we need to focus on the results of each process, as follows:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17-18

Import multiprocessing Import Time def func (msg): For I-xrange (3): Print msg time.sleep (1) return "Done" + msg if __na me__ = = "__main__": Pool = multiprocessing. Pool (processes=4) result = [] for i in xrange: msg = "Hello%d"% (i) result.append (Pool.apply_async (func, (msg,)) PO Ol.close () Pool.join () for res in Result:print res.get () print "sub-process (es) done."

I hope this article will help you with your Python programming.