Python multi-process concurrency (multiprocessing) Usage example detailed

Http://www.jb51.net/article/67116.htm

This example describes the Python multi-process concurrency (multiprocessing) usage. Share to everyone for your reference. The specific analysis is as follows:

Due to the limitations of Python design (I'm talking about the CPython we used). You can use up to 1 CPU cores.
Python provides a very useful multi-process package multiprocessing, you only need to define a function, Python will do everything else for you. With this package, you can easily convert from single-process to concurrent execution.

1. Create a single process

If we create a few new processes, we can do the following:

?

1234567891011

import multiprocessingimport timedef func(msg):  for i in xrange(3):    print msg    time.sleep(1)if __name__ == "__main__":  p = multiprocessing.Process(target=func, args=("hello", ))  p.start()  p.join()  print "Sub-process done."

2. Using Process Pools

Yes, you are not mistaken, not a thread pool. It lets you run over multicore CPUs, and it's easy to use.

Note that with Apply_async, if you fall into async, it becomes a blocking version.

Processes=4 is the maximum number of concurrent processes.

?

1234567891011121314

import multiprocessingimport timedef func(msg):  for i in xrange(3):    print msg    time.sleep(1)if __name__ == "__main__":  pool = multiprocessing.Pool(processes=4)  for i in xrange(10):    msg = "hello %d" %(i)    pool.apply_async(func, (msg, ))  pool.close()  pool.join()  print "Sub-process(es) done."

3, use pool, and need to pay attention to the results

More often than not, we need to focus not only on multi-process execution, but also on the execution results of each process, as follows:

?

123456789101112131415161718

import multiprocessingimport timedef func(msg):  for i in xrange(3):    print msg    time.sleep(1)  return "done " + msgif __name__ == "__main__":  pool = multiprocessing.Pool(processes=4)  result = []  for i in xrange(10):    msg = "hello %d" %(i)    result.append(pool.apply_async(func, (msg, )))  pool.close()  pool.join()  for res in result:    print res.get()  print "Sub-process(es) done."

Hopefully this article will help you with Python programming.

Python multi-process concurrency (multiprocessing) Usage example detailed