Try to learn Python's multi-process module, compared to multi-threading, the approximate difference is:
1. Multi-process processing faster
2. It is inconvenient to exchange data between various sub-processes of multiple processes
Multi-process invocation mode
Process basic use multicore ()
Process Pool Optimization process call Multicore_pool (), when using the process pool, the Operation Xu function has a return, and the basic usage of the process is not receive return
Process queue usage, most of the methods are consistent with Python's basic queue,
Q=mp. Queue () statement
Q.put () add
Q.get () release
Q.empty () judgment is not empty
"" "Created on Tue Dec 5 09:00:00 2017@author:hellcat" "" Import Timeimport NumPy as Npimport threading as Tdimport MULTIPR Ocessing as Mpdef Job (q): For I in range (3): Q.put (i) def multicore (): Q=mp. Queue () PS = [MP. Process (target=job,args= (q,)), MP. Process (target=job,args= (q,))] # Basic Child process creation method for p in Ps:p.start () Time.sleep (3) for P in PS: P.join () # need to reconcile end while True:if q.empty ()! = True:print ( Q.get ()) Else:breakdef multithread (): Q=mp. Queue () PS = [TD. Thread (target=job,args= (q,)), TD. Thread (target=job,args= (Q,))] # Basic Child thread creation method for p in Ps:p.start () Time.sleep (3) for P in PS: P.join () while True:if q.empty ()! = True:print (Q.get ()) Else:brea Kdef Job_pool (q): Res0 = 0 for I in range (Np.random.randint): Res0 + = i**3 return res0deF Multicore_pool (): Pool = MP. Pool (processes=2) # Initializes the process pool, you can specify the number of processes res1 = Pool.map (Job_pool,range (20)) # Map method Add multiple threads simultaneously in thread pool print (RES1) # return value for function res2 = [Pool.apply_async (Job _pool, (I,))) for I in range (20)] # Single add process in process pool print ([Res.get () for res in res2]) # Note at this point in each process's The return is not directly a return, and the Get method is required to get the return # for RES in Res2: # print (Res.get ()) if __name__== ' __main__ ': # Multicor E () # multithread () Multicore_pool ()
Process shared variables & process Locks
#!/usr/bin/env python3#-*-coding:utf-8-*-"" "Created on Tue Dec 5 10:15:58 2017@author:hellcat" "" Import Timeimport Multiprocessing as Mpdef Job (v,num,l): # The shared variable instance needs to be passed in as a parameter l.acquire () # process Lock Lock for _ in range (10): Time.sleep (0.1) v.value + = num # Gets the value of the shared variable and operates print (v.value) l.release () # process lock Release def muticore ( ): L = MP. Lock () # instantiates the process lock v = MP. Value (' I ', 0) # instantiates the shared variable # p = MP. Pool (processes=2) # P.map (Job, ((v,1,l), (v,3,l))) P1 = MP. Process (target=job,args= (v,1,l)) P2 = MP. Process (target=job,args= (v,3,l)) P1.start () P2.start () p1.join () p2.join () if __name__ = = ' __main__ ': muticore ()
Unlike multithreading, which uses global variables to share variables, multiple processes must use V = MP. The statement of Value (' I ', 0) declares the variable (which can actually be MP.) Array (' i ', [list,[[1,2]), note that this can only be 1-d]], and pass V as a parameter to the function, and use V.value to awaken its value when used inside the function.
A process lock prevents other processes from using shared variables when locked, so you can see that the output is executed 10 times +1 and then 10 times +3, without using process locks +1 and +3 without regular alternation:
Comment out the process lock:
1
4
5
8
9
12
13
16
17
20
21st
24
25
28
29
32
33
36
37
40
Use a process lock (the above code does not comment out the process lock):
1
2
3
4
5
6
7
8
9
10
13
16
19
22
25
28
31
34
37
40
"Python" multi-process processing