Python multi-process concurrency (multiprocessing) Usage example detailed

This example describes the Python multi-process concurrency (multiprocessing) usage. Share to everyone for your reference. The specific analysis is as follows:

Due to the limitations of Python design (I'm talking about the CPython we used). You can use up to 1 CPU cores.
Python provides a very useful multi-process package multiprocessing, you only need to define a function, Python will do everything else for you. With this package, you can easily convert from single-process to concurrent execution.

1. Create a single process

If we create a few new processes, we can do the following:

Import Multiprocessingimport timedef func (msg): For  I in Xrange (3):    print msg    time.sleep (1) if __name__ = = "_ _main__ ":  p = multiprocessing. Process (Target=func, args= ("Hello",))  P.start ()  p.join ()  print "sub-process done."

2. Using Process Pools

Yes, you are not mistaken, not a thread pool. It lets you run over multicore CPUs, and it's easy to use.

Note that with Apply_async, if you fall into async, it becomes a blocking version.

Processes=4 is the maximum number of concurrent processes.

Import Multiprocessingimport timedef func (msg): For  I in Xrange (3):    print msg    time.sleep (1) if __name__ = = "_ _main__ ":  pool = multiprocessing. Pool (processes=4)  for I in Xrange (Ten):    msg = "Hello%d"% (i)    Pool.apply_async (func, (msg,))  Pool.close ()  pool.join ()  print "sub-process (es) done."

3, use pool, and need to pay attention to the results

More often than not, we need to focus not only on multi-process execution, but also on the execution results of each process, as follows:

Import Multiprocessingimport timedef func (msg): For  I in Xrange (3):    print msg    time.sleep (1)  Return "Done" + msgif __name__ = = "__main__":  pool = multiprocessing. Pool (processes=4)  result = [] for  I in Xrange (Ten):    msg = "Hello%d"% (i)    result.append (pool.apply_ Async (func, (msg,)))  pool.close ()  pool.join () for  res in result:    print res.get ()  print " Sub-Process (es) done. "

Hopefully this article will help you with Python programming.

