Multiprocessing Module Introduction
Multithreading in Python does not take advantage of multicore advantages, and if you want to fully use the resources of multicore CPUs, Os.cpu_count can view the number of logical CPUs, most of which require multiple processes in Python, and Python provides multiprocessing
The Muliprocessing module is used to open sub-processes and perform our tasks (such as functions) in sub-processes, which
Threading Module Threading Programming Interface Similar
The multiprocessing module has many functions, supports sub-processes, communicates and shares data, performs different forms of synchronization, and provides components such as Process,queue,pipe,lock.
Multiprocessing Unlike threads, processes do not have any shared state and process modification data is limited to the process
Process class
Process (Group,target=func,name=str,args (x,), kwargs={' num ': 1,})
An object instantiated by the class that represents a task in a child process
Emphasis: Use of keywords to pass parameters to process
args Specifies the location parameter for the function that corresponds to target, which is a tuple form, and the bracket class must have a comma
Parameter description:
1 The group parameter is not used, and the value is always None2 3 target to represent the calling object, that is, the task that the child process is performing 4 5 args represents the location of the calling object, the tuple, args= ("Egon",) 6 7 Kwargs represents the dictionary of the calling object, kwargs={' Name ': ' Egon ', ' Age ': 18}8 9 name for child process
Method Description:
1.p.start () starts the process and calls the P.run () in the child process
The 2.p.run () process starts with a method that runs when he goes to call the target specified function, and we always have to implement the run () method in the custom/inherit and Process class
3.p.terminate () forcibly terminates the process P, does not perform any cleanup operations, and if P creates a child process, the subprocess becomes a zombie process
Using this method requires special care in this case, if P also holds a lock then it will not be released, resulting in a deadlock
4.p.is_alive () returns True if P is still running
5.p.join ([timeout]) main program waits for p to terminate (emphasis is that the main program is waiting, P is running), timeout is optional
Timeout, P.join () can only join the START process, placed under Start (), and cannot join the process of run open
Property Description:
P.deamon, the default value is False, if set to True, represents the daemon that P is running in the background, and when P parent process terminates, p also terminates with it, and
When set to True, p cannot create its own new process, P.deamon must be set before P.start ()
Name of the p.name process
ID of the P.PID process
Use of the Process class
Process () in Windo must be put to if__name__== ' __main__ ': run down, not called at import
Because window does not have fork, the multi-processing module launches a new Python process and imports the calling module
If you call process () on import, name will start a new process with infinite inheritance (or until the machine runs out of resources)
Two ways to create and open a child process
#开进程的方法一:
Import time
Import Random
From multiprocessing import Process
def piao (name):
Print ('%s piaoing '%name)
Time.sleep (Random.randrange (1,5))
Print ('%s Piao end '%name)
P1=process (target=piao,args= (' Egon ',)) #必须加, No.
P2=process (target=piao,args= (' Alex ',))
P3=process (target=piao,args= (' Wupeqi ',))
P4=process (target=piao,args= (' Yuanhao ',))
P1.start ()
P2.start ()
P3.start ()
P4.start ()
Print (' main thread ')
Method One
#开进程的方法二:
Import time
Import Random
From multiprocessing import Process
Class Piao (Process):
def __init__ (self,name):
Super (). __INIT__ ()
Self.name=name
def run (self):
Print ('%s piaoing '%self.name)
Time.sleep (Random.randrange (1,5))
Print ('%s Piao end '%self.name)
P1=piao (' Egon ')
P2=piao (' Alex ')
P3=piao (' Wupeiqi ')
P4=piao (' Yuanhao ')
P1.start () #start会自动调用run
P2.start ()
P3.start ()
P4.start ()
Print (' main thread ')
Method Two
The direct memory space of the process is isolated
Daemon process
Master Process Creation Daemon
One: The daemon terminates after the execution of the main process code is completed
Second: The daemon can no longer open the child process, or throw an exception: Assertionerror:daemonic processes is not allowed to has children
Note: Processes are independent of each other, the main process code is running and the daemon terminates
From multiprocessing import Process
Import time
Import Random
Class Piao (Process):
def __init__ (self,name):
Self.name=name
Super (). __INIT__ ()
def run (self):
Print ('%s is piaoing '%self.name)
Time.sleep (Random.randrange (1,3))
Print ('%s is Piao end '%self.name)
P=piao (' Egon ')
P.daemon=true #一定要在p. Start (), set p as daemon, prevent p from creating child processes, and the parent process Code execution ends, p terminates the run
P.start ()
Print (' master ')
Process Synchronization (Lock)
Data between processes is not shared, but sharing the same set of file systems, so access to the same file, or the same print terminal, is no problem,
And the share brings competition, the result of competition is disorder, how to control, is to add lock processing
#文件db的内容为: {"Count": 1}
#注意一定要用双引号, or JSON is not recognized
From multiprocessing import Process,lock
Import Time,json,random
Def search ():
Dic=json.load (Open (' db.txt '))
Print (' \033[43m remaining votes%s\033[0m '%dic[' count '])
Def get ():
Dic=json.load (Open (' db.txt '))
Time.sleep (0.1) #模拟读数据的网络延迟
If dic[' count '] >0:
dic[' Count ']-=1
Time.sleep (0.2) #模拟写数据的网络延迟
Json.dump (Dic,open (' Db.txt ', ' W '))
Print (' \033[43m ticket successful \033[0m ')
def task (lock):
Search ()
Lock.acquire ()
Get ()
Lock.release ()
if __name__ = = ' __main__ ':
Lock=lock ()
For I in range: #模拟并发100个客户端抢票
P=process (target=task,args= (lock,))
P.start ()
Locking: Ticket purchase behavior from concurrency to serial, sacrificing operational efficiency, but to ensure data security
#加锁可以保证多个进程修改同一块数据时, only one task at a time can be modified, that is, the serial modification, yes, the speed is slow, but at the expense of the speed to ensure the data security. Although you can use file-sharing data for interprocess communication, the problem is: 1. Inefficient (shared data is file-based and files are data on the hard disk) 2. Need to lock your own handle
Python Multi-process