Python concurrent programming Multi-process (Implementation)

I. Introduction of the Multipricessing module

Multithreading in Python does not take advantage of multicore advantages, if you want to fully use multi-core CPU resources, in Python most of the cases need multithreading, Python provides the multiprocessing module

The multiprocessing module is used to open sub-processes and perform our tasks (such as functions) in the subprocess, similar to the programming interface of the Multithreaded module threading class.

The multiprocessing module has many functions: supporting sub-processes, communicating and sharing data, performing different forms of synchronization, and providing components such as Process class, Queue class, Pipe class, lock class, etc.

Ii. Introduction to the Process module

Basic format:

1  fromMultiprocessingImportProcess#Import Module2 3 defFunc (x):#Define a function (a function that waits for a new thread to execute)4     Print(x)5 6 if __name__=='__main__':#You must add this sentence to run Windows7p = Process (target=func,args= ('Pass the reference',))#instantiate a process, pass the function name as a parameter, pass the argument (tuple form) that needs to be passed to the function8P.start ()#(notifies the operating system) to turn on this process

Called by the class:

 fromMultiprocessingImportProcess#Calling ModuleclassMyprocess (Process):#To define a class, you must inherit the process class    def __init__(self,name):#If you need parameters, you must have the Init methodSuper ().__init__()#If you have an Init method, be sure to call the parent class's Init methodSelf.name =namedefRun (self):#Be sure to implement a run method to override the run of the parent class        Print('Child process%s is turned on'%self.name)if __name__=='__main__': P= Myprocess ('AAA')#instantiating an object of a custom classP.start ()#Open for

Parameter description:

target represents the calling object, the task that the child process is to perform, and the args represents the positional parameter tuple of the calling object, such as: args= (1,) or args= (Kwargs), which represents the dictionary of the calling object, such as: Kwargs={ ' name ':'fuyong'}

Method Description:

P.start ()    starts the process and calls the run () method of the child process P.run () The      method that runs when the process starts, formally it goes to invoke the function specified by the target, and we customize the class to be sure to implement the method P.terminate () Force terminate the program p, no cleanup operation, if p creates a child process, then the child process is a ' zombie process             using this method to be particularly careful, if p also retains a lock, then the lock will not be released, resulting in deadlock p.is_alive () to determine whether P is still running ,     If you add this method, the main thread will wait until this thread finishes running after the run Returns Truep.join ().

Property Description:

P.daemon the default value is False, and if set to True, represents the daemon running in the background, when P's parent process terminates, p also terminates the PID of the P.name process name P.pid process

Third, the Guardian process

not = Truep.daemon The default value is False, and if set to True, represents the daemon that runs in the background, and when P's parent process terminates, p also terminates

V. Process synchronization (LOCK)

Data between processes is not shared, but sharing the same set of file systems, so access to the same file, or the same print terminal, is no problem,

And the share brings competition, the result of competition is disorder, how to control, is to add lock processing

Lock mode:

1. Import Lock Class

2. Instantiate a lock = Lock ()

3. Pass lock as parameter to child process function

4, the function in need of the shackles of code before the Lock.acquire () method, where the need to release the lock to add the Lock.release () method

Vi. queues

Processes are isolated from each other, and to implement interprocess communication (IPC), the Multiprocessing module supports two forms: queues and pipelines, both of which use message passing

 from  multiprocessing import   Queueq  = Queue (3) #   Limit up to 3 if more than 3 are blocked, you need to wait to take out 1 before you can continue to put  q.put ( 1) #   put data (can put any data type)  Q.put (2) #   put data (can put any data type)  Q.put (3) #   Put data (can put any data type)  print  (Q.get ()) #   take data  print  (Q.get ()) #   take data  print  (Q.get ( ) #   take data  

Python concurrent programming Multi-process (Implementation)