Python multi-process related pits

Python's multiprocessing module implements multi-process functionality, but there are only a few simple uses of the official documentation, mainly the use of functions as the target of process, and how to use multiple processes in class without much explanation. Google out of two more detailed articles, it is recommended to get started from them:

Https://pymotw.com/2/multiprocessing/basics.html

Https://pymotw.com/2/multiprocessing/communication.html

Here's a look at the pit that I've encountered this week on Python's many processes:

When you create a process, the parameters must be pickle, so some custom class object instances cannot be used as arguments.

Unlike threading, the multiprocessing process parameter must be able to be serialized by Pickle

Python 2.7,can ' t pickle <type ' instancemethod ' >

Python version 2.7 of Python 3.5, multiprocessing behavior is different, some code can be run in 3.5, in 2.7 but run error

For example, you can run in 3.5, because in version 3.5, pick can serialize more types.

Try to avoid including multiprocess in class instances. Manager instance, or there will be

Typeerror:pickling an Authenticationstring object was disallowed for security reasons

Or:

_pickle. Picklingerror:can ' t pickle <class ' weakref ';: Attribute lookup weakref on Builtins failed inter-process shared objects, managed with manager

The manager generates a process, so accessing the uniform variable between different processes is done through the IPC and there is a performance overhead.

About the file code where the main process resides

When using multiprocessing, the main module is import into each process, so the part of the child process must be created using the

if __name__ = = ' __main__:

To be protected, otherwise there will be runtime error or recursive creation of child processes

Python multi-process related pits