Python Full Stack Development Foundation "21st" mutex and three communication modes (IPC) between processes and producer consumer models

One, mutual exclusion lock

Data is isolated between processes, but sharing a set of file systems allows direct communication of the process through files, but the problem is that it has to be locked up for itself.

Note: The purpose of the lock is to ensure that multiple processes modify the same piece of data, the same time can only have one modification, that is, the serial modification, yes, the speed is slow, sacrificing speed and ensure the data security.

1. Toilet Small Example: You must lock the door when you go to the bathroom, someone came to see the door locked, will wait outside, waiting for you to open the door when the next talent to the toilet.

From multiprocessing import Process,lockimport osimport timedef work (mutex):    mutex.acquire ()    print (' task[%s] Potty '%os.getpid ())    Time.sleep (3)    print (' task[%s] on the toilet '%os.getpid ())    mutex.release () if __name__ = = ' __ Main__ ':    mutex = Lock ()    p1 = Process (target=work,args= (mutex))    P2 = Process (target=work,args= (mutex))    p3 = Process (target=work,args= (mutex))    P1.start ()    P2.start () P3.start ()    p1.join ()    P2.join ()    p3.join ()    print (' master ')

2. Mock grab ticket (also using the principle of mutual exclusion Lock: Lock Mutex)

Import jsonimport timeimport randomimport osfrom multiprocessing import process,lockdef Chakan ():    dic = Json.load ( Open (' Piao ',)  # Check the number of votes first, that is, open the file    print (' remaining votes:%s '% dic[' count ')  # View the remaining votes def buy ():    dic = Json.load ( Open (' Piao ',))    if dic[' count ']>0: #如果还有票        dic[' count ']-=1 #就修改里面的值-1        time.sleep (Random.randint (1,3) #执行里面买票的一系列操作就先不执行了, let's sleep for a while (and sleep randomly)        json.dump (Dic,open (' Piao ', ' W '))        print ('%s ticket successfully purchased '% Os.getpid ())  # Current ID Ticket Success def task (mutex): #抢票    Chakan ()  #因为查看的时候大家都可以看到, no need to lock    mutex.acquire () #加锁    buy () # Buy the time must be one of the buy, first wait for a person to buy finished, the people behind the buy    mutex.release () #取消锁if __name__ = = ' __main__ ':    mutex = Lock () for    I in Range: #让50个人去访问那个票数        p = Process (target=task,args= (mutex))        P.start ()

　　

Second, other properties of the process object

P.daemon: Daemon (must set daemon before opening): If the parent process dies, the child process P is dead, too.

P.join: The parent process, such as P executes before running the main process, is the parent process blocked in place, and P is still running in the background.

Terminate: Forced shutdown. (Make sure there are no other sub-processes in P when closed, if there is a child process, you can use this method to force the shutdown will produce a zombie process (for example: if you hang up, you haven't hung, then no one to your corpse, ah haha))

Is_alive: When the process is closed, it will not be closed immediately, so the results is_alive immediately view may still be alive

P.join (): Parent process at the end of equal p, is the parent process is blocked in place, and P is still running in the background

P.name: View Name

P.pid: View ID

We can briefly introduce the zombie process:

The child process runs complete, but the parent process is not recycled, and the child process does not actually exit, it still occupies the system resources, so the process is called the zombie process .

Because the zombie process resources have not been recycled, resulting in a waste of system resources, excessive zombie process will cause system performance degradation, so should avoid the zombie process.

From multiprocessing import Processimport osimport timedef work ():    print ('%s is working '%os.getpid ())    Time.sleep (3) if __name__ = = ' __main__ ':    p1 =process (target=work)    p2 =process (target=work) P3 =process    ( target=work)    # P1.daemon = True    # P2.daemon = True #守护进程 (Guardian of his father)    # P3.daemon = True  # The main process died the child process also died (would not execute the subprocess)    P1.start () P2.start () P3.start () p3.join ()    p2.join (    )    P1.join () #多个join就是在等花费时间最长的那个运行完就执行主程序了    print (' main program ') #-Understanding Method---------------#     P1.terminate ()  # Force shutdown Process #     Time.sleep (3) #     print (p1.is_alive ())  #看是不是还活着 #     print (p1.name) #查看进程名字 #     Print ( p1.pid) #查看id号 #     Print (' main program ')

Iii. three communication (IPC) modes between processes:

Way One: Queue (recommended)

Processes are isolated from each other, and to implement interprocess communication (IPC), the Multiprocessing module supports two forms: queues and pipelines, both of which use message passing

1. Queue: Queue similar to a pipeline, element FIFO
One thing to note is that the queue is in-memory, the process exits, the queue is emptied, and the queue is a blocking pattern.
2. Queue classification
There are many kinds of queues, but all depend on the module queue
Queue. Queue ()  #先进先出
Queue. Lifoqueue () #后进先出
Queue. Priorityqueue () #优先级队列
Queue.deque () #双线队列

Create a queue class (the underlying is implemented as a pipe and lock):

Queue ([MaxSize]): Creates a shared process queue, which is a multi-process secure queue that enables data transfer between multiple processes using the queue.

Parameter description:

MaxSize is the maximum number of items allowed in a queue, and no size limit is omitted.

Method Description:

The Q.put method is used to insert data into the queue, and the put method has two optional parameters: Blocked and timeout. If blocked is true (the default) and timeout is a positive value, the method blocks the time specified by timeout until the queue has the remaining space. If timed out, a Queue.full exception is thrown. If blocked is false, but the queue is full, an Queue.full exception is thrown immediately. The Q.get method can read from the queue and delete an element. Similarly, the Get method has two optional parameters: Blocked and timeout. If blocked is true (the default) and timeout is a positive value, then no element is taken within the wait time, and a Queue.empty exception is thrown. If blocked is false, there are two cases where the queue has a value that is available, and the value is immediately returned, otherwise the Queue.empty exception is thrown immediately if it is empty.  Q.get_nowait (): Same as Q.get (false) q.put_nowait (): Same as Q.put (false) Q.empty (): When calling this method, q is null to return true and the result is not reliable, for example, in the process of returning true, If the item is added to the queue. Q.full (): When this method is called, Q is full to return true, and the result is unreliable, for example, in the process of returning true, if the items in the queue are taken away. Q.qsize (): Returns the correct number of current items in the queue, and the results are unreliable, for the same reason as Q.empty () and Q.full ()

Application:

#队列 # 1. You can put any type of # 2 into the queue. FIFO from multiprocessing import process,queueq= Queue (3) q.put (' first ')  #默认block =trueq.put (' second ') Q.put (' third ') print (Q.get ()) print (Q.get ()) print (Q.get ())

　　

Python Full Stack Development Foundation "21st" mutex and three ways of communication (IPC) between processes and producer consumer models