Python Multi-process

Source: Internet
Author: User

Multiprocessing Module Introduction

Multithreading in Python does not take advantage of multicore advantages, and if you want to fully use the resources of multicore CPUs, Os.cpu_count can view the number of logical CPUs, most of which require multiple processes in Python, and Python provides multiprocessing

The Muliprocessing module is used to open sub-processes and perform our tasks (such as functions) in sub-processes, which

Threading Module Threading Programming Interface Similar

The multiprocessing module has many functions, supports sub-processes, communicates and shares data, performs different forms of synchronization, and provides components such as Process,queue,pipe,lock.

Multiprocessing Unlike threads, processes do not have any shared state and process modification data is limited to the process

Process class

Process (Group,target=func,name=str,args (x,), kwargs={' num ': 1,})

An object instantiated by the class that represents a task in a child process

Emphasis: Use of keywords to pass parameters to process

args Specifies the location parameter for the function that corresponds to target, which is a tuple form, and the bracket class must have a comma

Parameter description:

1 The group parameter is not used, and the value is always None2 3 target to represent the calling object, that is, the task that the child process is performing 4 5 args represents the location of the calling object, the tuple, args= ("Egon",) 6 7 Kwargs represents the dictionary of the calling object, kwargs={' Name ': ' Egon ', ' Age ': 18}8 9 name for child process


Method Description:

1.p.start () starts the process and calls the P.run () in the child process
The 2.p.run () process starts with a method that runs when he goes to call the target specified function, and we always have to implement the run () method in the custom/inherit and Process class
3.p.terminate () forcibly terminates the process P, does not perform any cleanup operations, and if P creates a child process, the subprocess becomes a zombie process
Using this method requires special care in this case, if P also holds a lock then it will not be released, resulting in a deadlock
4.p.is_alive () returns True if P is still running
5.p.join ([timeout]) main program waits for p to terminate (emphasis is that the main program is waiting, P is running), timeout is optional
Timeout, P.join () can only join the START process, placed under Start (), and cannot join the process of run open

Property Description:

P.deamon, the default value is False, if set to True, represents the daemon that P is running in the background, and when P parent process terminates, p also terminates with it, and
When set to True, p cannot create its own new process, P.deamon must be set before P.start ()

Name of the p.name process

ID of the P.PID process


Use of the Process class

Process () in Windo must be put to if__name__== ' __main__ ': run down, not called at import

Because window does not have fork, the multi-processing module launches a new Python process and imports the calling module
If you call process () on import, name will start a new process with infinite inheritance (or until the machine runs out of resources)

Two ways to create and open a child process

#开进程的方法一:
Import time
Import Random
From multiprocessing import Process
def piao (name):
Print ('%s piaoing '%name)
Time.sleep (Random.randrange (1,5))
Print ('%s Piao end '%name)

P1=process (target=piao,args= (' Egon ',)) #必须加, No.
P2=process (target=piao,args= (' Alex ',))
P3=process (target=piao,args= (' Wupeqi ',))
P4=process (target=piao,args= (' Yuanhao ',))

P1.start ()
P2.start ()
P3.start ()
P4.start ()
Print (' main thread ')

Method One

#开进程的方法二:
Import time
Import Random
From multiprocessing import Process


Class Piao (Process):
def __init__ (self,name):
Super (). __INIT__ ()
Self.name=name
def run (self):
Print ('%s piaoing '%self.name)

Time.sleep (Random.randrange (1,5))
Print ('%s Piao end '%self.name)

P1=piao (' Egon ')
P2=piao (' Alex ')
P3=piao (' Wupeiqi ')
P4=piao (' Yuanhao ')

P1.start () #start会自动调用run
P2.start ()
P3.start ()
P4.start ()
Print (' main thread ')

Method Two

The direct memory space of the process is isolated

Daemon process

Master Process Creation Daemon

One: The daemon terminates after the execution of the main process code is completed

Second: The daemon can no longer open the child process, or throw an exception: Assertionerror:daemonic processes is not allowed to has children

Note: Processes are independent of each other, the main process code is running and the daemon terminates

From multiprocessing import Process
Import time
Import Random

Class Piao (Process):
def __init__ (self,name):
Self.name=name
Super (). __INIT__ ()
def run (self):
Print ('%s is piaoing '%self.name)
Time.sleep (Random.randrange (1,3))
Print ('%s is Piao end '%self.name)


P=piao (' Egon ')
P.daemon=true #一定要在p. Start (), set p as daemon, prevent p from creating child processes, and the parent process Code execution ends, p terminates the run
P.start ()
Print (' master ')

Process Synchronization (Lock)

Data between processes is not shared, but sharing the same set of file systems, so access to the same file, or the same print terminal, is no problem,

And the share brings competition, the result of competition is disorder, how to control, is to add lock processing

#文件db的内容为: {"Count": 1}
#注意一定要用双引号, or JSON is not recognized
From multiprocessing import Process,lock
Import Time,json,random
Def search ():
Dic=json.load (Open (' db.txt '))
Print (' \033[43m remaining votes%s\033[0m '%dic[' count '])

Def get ():
Dic=json.load (Open (' db.txt '))
Time.sleep (0.1) #模拟读数据的网络延迟
If dic[' count '] >0:
dic[' Count ']-=1
Time.sleep (0.2) #模拟写数据的网络延迟
Json.dump (Dic,open (' Db.txt ', ' W '))
Print (' \033[43m ticket successful \033[0m ')

def task (lock):
Search ()
Lock.acquire ()
Get ()
Lock.release ()
if __name__ = = ' __main__ ':
Lock=lock ()
For I in range: #模拟并发100个客户端抢票
P=process (target=task,args= (lock,))
P.start ()

Locking: Ticket purchase behavior from concurrency to serial, sacrificing operational efficiency, but to ensure data security

#加锁可以保证多个进程修改同一块数据时, only one task at a time can be modified, that is, the serial modification, yes, the speed is slow, but at the expense of the speed to ensure the data security. Although you can use file-sharing data for interprocess communication, the problem is: 1. Inefficient (shared data is file-based and files are data on the hard disk) 2. Need to lock your own handle

Python Multi-process

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.