Multi-process in Python concurrent programming (code implementation)

I. Introduction of the Multipricessing module

Multithreading in Python does not take advantage of multicore advantages, if you want to fully use multi-core CPU resources, in Python most of the cases need multithreading, Python provides the multiprocessing module

The multiprocessing module is used to open sub-processes and perform our tasks (such as functions) in the subprocess, similar to the programming interface of the Multithreaded module threading class.

The multiprocessing module has many functions: supporting sub-processes, communicating and sharing data, performing different forms of synchronization, and providing components such as Process class, Queue class, Pipe class, lock class, etc.

Ii. Introduction to the Process module

Basic format:

1 from multiprocessing import Process         #导入模块2 3 def func (x):                                #定义一个函数 (function waiting to open a new thread) 4     print (x) 5 6 If __name__ = = ' __main__ ':                  #windows下一定要加上这句话才能运行7     p = Process (target=func,args= (' pass parameter ')) #实例化一个进程, passing the function name as a parameter, Pass a parameter (tuple form) that needs to pass to the function 8     p.start ()                               # (notifies the operating system) to turn on this process

Called by the class:

From multiprocessing import process     #调用模块class myprocess (process):               #定义一个类, you must inherit the process class    def __init__ ( Self,name):            #如果需要参数, be sure to have the Init method        super (). __init__ ()              #如果有init方法, be sure to call the parent class's Init method        Self.name = Name    def run (self):                      #一定要实现一个run方法来重写父类的run        Print (' child process%s turned on '%self.name) if __name__ = = ' __main__ ':    p = myprocess (' aaa ')                #实例化一个自定义类的对象    p.start ()                           #开启进行

Parameter description:

The target represents the calling object, the task that the child process is to perform, and the args represents the location of the calling object as a tuple of parameters, such as: args= (1,) or args= (Kwargs), which represents the dictionary of the calling object, such as: kwargs={' name ': ' Fuyong '}

Method Description:

P.start ()    starts the process and calls the run () method of the child process P.run () The      method that runs when the process starts, formally it goes to invoke the function specified by the target, and we customize the class to be sure to implement the method P.terminate () Force terminate the program p, no cleanup operation, if p creates a child process, then the child process is a ' zombie process             using this method to be particularly careful, if p also retains a lock, then the lock will not be released, resulting in deadlock p.is_alive () to determine whether P is still running, On Run return Truep.join ()     If this method is added, the main thread will wait until this thread finishes running

Property Description:

P.daemon the default value is False, and if set to True, represents the daemon running in the background, when P's parent process terminates, p also terminates the PID of the P.name process name P.pid process

Third, the Guardian process

Master Process Creation Daemon One: The daemon terminates after the execution of the main process code: The daemon can no longer open the child process, or throws an exception: Assertionerror:daemonic processes is not allowed to has chil dren Note: The process is independent of each other, the main process code run ends, the daemon terminates the method of setting the daemon: P.daemon = Truep.daemon The default value is False, if set to True, represents the daemon running in the background, when P's parent process terminates, P will also be terminated with the

V. Process synchronization (LOCK)

Data between processes is not shared, but sharing the same set of file systems, so access to the same file, or the same print terminal, is no problem,

And the share brings competition, the result of competition is disorder, how to control, is to add lock processing

Lock mode:

1. Import Lock Class

2. Instantiate a lock = Lock ()

3. Pass lock as parameter to child process function

4, the function in need of the shackles of code before the Lock.acquire () method, where the need to release the lock to add the Lock.release () method

Vi. queues

Processes are isolated from each other, and to implement interprocess communication (IPC), the Multiprocessing module supports two forms: queues and pipelines, both of which use message passing

From multiprocessing Import Queueq = Queue (3)  #限制最多放3个  If more than 3 are blocked and need to wait to take out 1 before continuing to put Q.put (1)      # Put data (can put any data type) Q.put (2)      #放数据 (can put any data type) Q.put (3)      #放数据 (can put any data type) print (Q.get ()) #拿数据print (Q.get ()) # Take data print (Q.get ()) #拿数据

Multi-process in Python concurrent programming (code implementation)