Python: Process

3 Ways to implement multi-tasking:

Multi-process Mode

Multithreaded mode

Multi-process + multithreaded mode

A thread is the smallest execution unit, and a process consists of at least one thread.

Multi-process (multiprocessing)

Unix/linux with the fork () function, the call returns two values at a time, and the child process returns 0 forever, and the process returns the ID of the child process.

The child process calls the Getppid () function to get the ID of the husband process.

Unix/linux Creating child processes

ImportOSPrint("Process (%s) Start ..."%Os.getpip ()) PID= Os.fork ()#no fork calls on WindowsifPID = =0:Print("I AM Child process (%s) and my parent is%s"%(Os.getpid (), Os.getppid ()))Else:    Print("I (%s) just created a child process (%s)"% (Os.getpid (), PID))

Multiprocessing: Cross-platform multi-process

The multiprocessing module provides a process class to represent a processing object

 fromMultiprocessingImportProcessImportOSdefRun_proc (name):Print("Run Child process%s (%s)"%(name, Os.getpid ()))if __name__=="__main__":    Print("Parent process%s."%os.getpid ()) P= Process (Target=run_proc, args= ('Test',))#Create a child process, pass in a parameter that executes a function and executes a function    Print("Child process would start.") P.start ()#Start child processP.join ()#wait for the child process to end and continue down for synchronization between processes    Print("Child process end.")

Execution Result:

Parent process 928. Process would start. Run Child process Test (929) ... Process end.

Pool: Process pool, batch creation of child processes

 fromMultiprocessingImportPoolImportOS, time, RandomdefLong_time_task (name):Print("Run Task%s (%s) ..."%(name, Os.getpid ())) Start=time.time () time.sleep (Random.random ()*) End=time.time ()Print("Task%s runs%0.2f seconds."% (name, (end-start )))if __name__=="__main__":    Print("Parent process%s."%os.getpid ()) P= Pool (4)     forIinchRange (5): P.apply_async (long_time_task, args=(i,))Print("waiting-subprocess done ...") P.close () P.join ()Print("All subprocess is done .")

Execution Result:

Parent process 4756 for all subprocess done ... Run Task 0 (39201 (16882 (29963 (31321 runs 0.324 (16882 runs 0.613 runs 1.414 runs 2.262.89 seconds . All subprocess is done.

Analysis:

Invoking a method on an Pool object join() waits for all child processes to complete before the call must be called before the call join() close() close() can continue to add new Process .

Note that the result of the output, task 0 ,, 1 2 3 is executed immediately, and the task 4 waits for a previous task to complete before it executes, because Pool the default size on my computer is 4, so that up to 4 processes are executed at the same time. This is a Pool deliberate limitation, not a limitation of the operating system

Child process (subprocess)

subprocessThe module allows us to start a subprocess very conveniently and then control its input and output

Import subprocess # The following example shows how to run the command nslookup www.python.org in Python code, which works just like the command line: Print ("$ nslookup www.python.rog"= Subprocess.call (["nslookup " " www.python.org " ])print("Exit Code:", R)

Operation Result:

$ nslookup www.python.orgServer:         192.168.19.4Address:    192.168.19.4#Non-authoritative answer: www.python.org    = python.map.fastly.net.Name:    199.27.79.223Exit code:0

If the child process also requires input, it can be entered using the Communicate () method:

ImportsubprocessPrint('$ nslookup') P= subprocess. Popen (['nslookup'], stdin=subprocess. PIPE, Stdout=subprocess. PIPE, stderr=subprocess. PIPE) output, err= P.communicate (b'Set q=mx\npython.org\nexit\n')Print(Output.decode ('Utf-8'))Print('Exit Code:', P.returncode)

Inter-process communication

ProcessThere is definitely a need for communication, and the operating system provides many mechanisms for communicating between processes. Python's multiprocessing modules wrap the underlying mechanism, providing, and Queue Pipes many other ways to exchange data

As an Queue example, create two sub-processes in the parent process, one to Queue write the data, and one to read the data from the Queue inside:

 fromMultiprocessingImportProcess, QueueImportOS, time, Random#code to write the data Process execution:defWrite (q):Print('Process to write:%s'%os.getpid ()) forValueinch['A','B','C']:        Print('Put%s to queue ...'%value) q.put (value) time.sleep (Random.random ())#read the code that the data process executes:defRead (q):Print('Process to read:%s'%os.getpid ()) whileTrue:value=q.get (True)Print('Get%s from queue.'%value)if __name__=='__main__':    #The parent process creates a queue and passes it to each child process:Q =Queue () PW= Process (Target=write, args=(q,)) PR= Process (Target=read, args=(q,))#Start child process PW, write:Pw.start ()#start child process PR, read:Pr.start ()#wait for PW to end:Pw.join ()#PR process is a dead loop, can not wait for its end, can only forcibly terminate:Pr.terminate ()

Execution Result:

Process to write:38724756 from the from from queue.

Python: Process