Python Multi-process programming (i)

Multiprocessing Module Introduction

Multithreading in Python does not take advantage of multicore advantages, and if you want to fully use the resources of multicore CPUs (Os.cpu_count () Viewing), most of the situations in Python require multiple processes to be used. Python provides the multiprocessing.
The multiprocessing module is used to open sub-processes and perform our custom tasks (such as functions) in the subprocess, which is similar to the programming interface of the Multithreaded module threading.

The multiprocessing module has many functions: supporting sub-processes, communicating and sharing data, performing different forms of synchronization, and providing components such as process, Queue, Pipe, lock, etc.

One thing that needs to be emphasized again is that, unlike threads, processes do not have any shared state, the process modifies data, and changes are limited to that process.

Introduction to the Process class

To create a process class:

Process([group [, target [, name [, args [, kwargs]]]]])，由该类实例化得到的对象，
表示一个子进程中的任务（尚未启动）
强调：
1. 需要使用关键字的方式来指定参数
2. args指定的为传给target函数的位置参数，是一个元组形式，必须有逗号

Parameter description:

group参数未使用，值始终为None
target表示调用对象，即子进程要执行的任务
args表示调用对象的位置参数元组，args=(1,2,‘egon‘,)
kwargs表示调用对象的字典,kwargs={‘name‘:‘egon‘,‘age‘:18}
name为子进程的名称

Method Description:

p.start()：启动进程，并调用该子进程中的p.run() 
p.run():进程启动时运行的方法，正是它去调用target指定的函数，我们自定义类
的类中一定要实现该方法  
p.terminate():强制终止进程p，不会进行任何清理操作，如果p创建了子进程，
该子进程就成了僵尸进程，使用该方法需要特别小心这种情况。如果p还保存了一
个锁那么也将不会被释放，进而导致死锁
p.is_alive():如果p仍然运行，返回True
p.join([timeout]):主线程等待p终止（强调：是主线程处于等的状态，
而p是处于运行的状态）。timeout是可选的超时时间，需要强调的是，p.join只能
join住start开启的进程，而不能join住run开启的进程

Property Description:

p.daemon：默认值为False，如果设为True，代表p为后台运行的守护进程，当p的
父进程终止时，p也随之终止，并且设定为True后，p不能创建自己的新进程，
必须在p.start()之前设置
p.name:进程的名称
p.pid：进程的pid
p.exitcode:进程在运行时为None、如果为–N，表示被信号N结束(了解即可)
p.authkey:进程的身份验证键,默认是由os.urandom()随机生成的32字符的字符串。
这个键的用途是为涉及网络连接的底层进程间通信提供安全性，这类连接只有在具有
相同的身份验证键时才能成功（了解即可）

  use of the process class

Note: the process () in Windows must be placed in # if __name__ = = ' __main__ ': under

Two ways to create and open a child process

#开进程的方法一:
import time
import random
from multiprocessing import Process
def piao(name):
   print(‘%s piaoing‘ %name)
   time.sleep(random.randrange(1,5))
   print(‘%s piao end‘ %name)
p1=Process(target=piao,args=(‘zhangsan‘,)) #必须加,号
p2=Process(target=piao,args=(‘lisi‘,))
p3=Process(target=piao,args=(‘wangwu‘,))
p4=Process(target=piao,args=(‘zhaoliu‘,))
p1.start()
p2.start()
p3.start()
p4.start()
print(‘主线程‘)

#开进程的方法二:
Import time
 import random 
  from multiprocessing   import process 
   class   Piao  (Process): 
       def   __init__  (self,name): 
        super (). __init__ () 
        self.name=name 
       def   run  (self): 
        print ( '%s piaoing '  %self.name) 
 
        time.sleep (Random.randrange ( 1,  5)) 
        print (  '%s Piao end '  %self.name) 
 P1=piao ( ' Zhangsan ') 
 P2=piao ( ' Lisi ') 
 P3=piao ( ' Wangwu ') 
 P4=piao ( ' Zhaoliu ') 
 P1.start ()    #start会自动调用run 
 P2.start () 
 P3.start () 
 P4.start () 
 Print ( ' main thread ')  

Join method for Process object

Master process, etc., waiting for the child process to end

from multiprocessing import Process
import time
import random

class Piao(Process):
   def __init__(self,name):
       self.name=name
       super().__init__()
   def run(self):
       print(‘%s is piaoing‘ %self.name)
       time.sleep(random.randrange(1,3))
       print(‘%s is piao end‘ %self.name)
p=Piao(‘zangsan‘)
p.start()
p.join(0.0001) #等待p停止,等0.0001秒就不再等了
print(‘开始‘)

With join, the program is not serial it???

From multiprocessing import Process
Import time
Import Random
def piao (name):
Print ('%s is piaoing '%name)
Time.sleep (Random.randint (1,3))
Print ('%s is Piao end '%name)

P1=process (target=piao,args= (' Zhangsan ',))
P2=process (target=piao,args= (' Lisi ',))
P3=process (target=piao,args= (' Wangwu ',))
P4=process (target=piao,args= (' Zhaoliu ',))

P1.start ()
P2.start ()
P3.start ()
P4.start ()

#有的同学会有疑问: Since join is waiting for the process to end, then I write like this, the process does not become serial again?
#当然不是了, it must be clear: P.join () is Who to wait?
#很明显p. Join () is to let the main thread wait for the end of P, stuck is the main thread and not the process P,

#详细解析如下:
#进程只要start就会在开始运行了, so P1-p4.start (), there are already four concurrent processes in the system.
#而我们p1. Join () is at the end of the P1, yes P1 as long as it does not end the main thread will remain stuck in place, which is the key to the problem
#join是让主线程等, while P1-P4 is still executing concurrently, P1.join, the rest of the P2,P3,P4 is still running, and so on #p1. Join ends, maybe p2,p3,p4 is over, so p2.join, P3.join.p4.join directly through the test without waiting
# So the total time spent on 4 joins is still the longest time that the process runs
P1.join ()
P2.join ()
P3.join ()
P4.join ()
Print (' main thread ')
#上述启动进程与join进程可以简写为
# P_L=[P1,P2,P3,P4]
#
# for p in p_l:
# P.start ()
#
# for p in p_l:
# p.join ()                  

Name and PID

From multiprocessingImport Process
Import time
Import Random
ClassPiao(Process):
  def   __init__  (self,name): 
          # self.name=name 
          # Super (). __init__ () #Process的__init__方法会执行self. Name=piao-1, 
          #                     #所以加到这里 will cover our self.name=name 
   &NBSP ;   &NBSP  #为我们开启的进程设置名字的做法 
        super (). __init__ () 
         Self.name=name 
       def   run  (self): 
        print (  '%s is piaoing '  %self.name) 
        time.sleep (Random.randrange ( 1,  3)) 
        print (  '%s is Piao end '  %self.name) 
 P=piao ( ' Zhangsan ') 
 P.start () 
 Print ( ' start ') 
 Print (p.pid)    #查看pid   

Daemon process

Master Process Creation Daemon

One: The daemon terminates after the execution of the main process code is completed

Second: The daemon can no longer open the child process, or throw an exception: Assertionerror:daemonic processes is not allowed to has children

Note: Processes are independent of each other, the main process code is running and the daemon terminates

from multiprocessing import Process
import time
import random

class Piao(Process):
   def __init__(self,name):
       self.name=name
       super().__init__()
   def run(self):
       print(‘%s is piaoing‘ %self.name)
       time.sleep(random.randrange(1,3))
       print(‘%s is piao end‘ %self.name)
p=Piao(‘zhangsan‘)
p.daemon=True #一定要在p.start()前设置,设置p为守护进程,禁止p创建子进程,并且父进程代码执行结束,p即终止运行
p.start()
print(‘主‘)

#主进程代码运行完毕,守护进程就会结束,迷惑人的例子
from multiprocessing import Process
from threading import Thread
import time
def foo():
   print(123)
   time.sleep(1)
   print("end123")
def bar():
   print(456)
   time.sleep(3)
   print("end456")
p1=Process(target=foo)
p2=Process(target=bar)
p1.daemon=True
p1.start()
p2.start()
print("main-------") #打印该行则主进程代码结束,则守护进程p1应该被终止,可能会有p1任务执行的打印信息123,因为主进程打印main----时,p1也执行了,但是随即被终止

Identify the QR code in the chart, and welcome to the Python Treasure Book

Python Multi-process programming (i)