Daemon, mock grab ticket example, mutex, semaphore, queue summary

Daemon process

Master Process Creation Daemon

One: The daemon terminates after the execution of the main process code is completed

Second: The daemon can no longer open the child process, or throw an exception: Assertionerror:daemonic processes is not allowed to has children

Note: Processes are independent of each other, the main process code is running and the daemon terminates

# daemon from multiprocessing import processimport os,time,randomdef task ():    print ('%s is running '%os.getpid ())    Time.sleep (2)    print ('%s is done '%os.getpid ()) If __name__ = = ' __main__ ':    p = Process (target=task)    P.daemon = True  # must precede P.start (), the daemon cannot turn on child process    P.start ()    print (' master ') ' to illustrate the daemon's scenario:    assume that there are two tasks to be done, to play the concurrency effect, Using a process allows the main process to execute a task    and then open a child process to perform a task.    If these two tasks have nothing to do, then open a child process directly if the task of the    master process does not have the meaning of the task of the child process, then the child process should    be set as the Daemon "' # # # of deluded people before it is opened. The main process code is finished, the daemon ends from multiprocessing import processfrom threading import Threadimport timedef foo ():    print (123)    time.sleep (1)    print ("end123") def Bar ():    print (456)    Time.sleep (3)    print ("end456") if __ name__ = = ' __main__ ':    p1 = Process (target=foo)    P2 = Process (target=bar)    P1.daemon = True    P1.start ( )    P2.start () print    ("main-------") # Prints the line then the main process code ends, the daemon P1 should be terminated, there may be a P1 task to execute the print information 123, because the main process prints the main----, The P1 was executed, but it was immediately terminated.

Daemon Threads

For the main thread, it means that all the non-daemon threads in the process of the main thread are running, and the main thread is running complete.

# daemon: Wait until all non-daemons in the process run out of threading import Threadimport os,time,randomdef Task ():    print ('%s is running '%o S.getpid ())    Time.sleep (2)    print ('%s is Do '%os.getpid ()) If __name__ = = ' __main__ ':    t = Thread (target= Task)    T.daemon = True  # must be preceded by P.start ()    t.start ()    print (' master ') # Confusing example from multiprocessing import Processfrom Threading Import Threadimport timedef foo ():    print (123)    Time.sleep (1)    print ("end123") def Bar ():    print (456)    Time.sleep (3)    print ("end456") if __name__ = = ' __main__ ':    t1 = Thread (target= Foo)    t2 = Thread (target=bar)    T1.daemon = True    t1.start ()    t2.start ()    print ("main-------")

Mutual exclusion Lock

# process from multiprocessing import Process,lockimport os,time,randomdef task (Mutex):    mutex.acquire ()    print ('%s Print 1 '%os.getpid ())    Time.sleep (Random.randint (1,3))    print ('%s Print 2 '%os.getpid ())    mutex.release () if __name__ = = ' __main__ ':    mutex = Lock ()    p1 = Process (target=task,args= (mutex))    P2 = Process (Target=task, Args= (mutex,))    p3 = Process (target=task,args= (mutex)) P1.start () P2.start () P3.start ()    # thread from Threading Import Thread,lockimport timen = 100def task ():    # Global N    # Mutex.acquire ()    # temp = n    # ti Me.sleep (0.1)    # n = temp-1    # mutex.release ()    global n with    mutex:        temp = n        time.sleep ( 0.1)        n = temp-1if __name__ = = ' __main__ ':    mutex = Lock ()    t_l = [] for    i in range:        t = thre AD (Target=task)        t_l.append (t)        T.start () for    T in t_l:        t.join ()    print (n)

Lock can ensure that multiple processes modify the same piece of data, only one task can be modified at the same time, that is, serial modification, yes, the speed is slow, but at the expense of speed to ensure the data security. Although you can use file sharing data to achieve interprocess communication, the problem is:

1. Inefficient (shared data is file-based, and files are data on the hard disk)

2. Need to lock the handle yourself

So we'd better find a solution that can take care of: 1, high efficiency (multiple processes sharing a piece of memory data) 2, help us handle the lock problem. This is the message-based IPC communication mechanism that the Mutiprocessing module provides for us: queues and pipelines.

1 both the queue and the pipeline are storing the data in memory

The 2 queue is also based on (pipe + lock) implementation, allowing us to escape from the complex lock problem

We should try to avoid using shared data, use messaging and queues whenever possible, avoid complex synchronization and locking problems, and often get better malleable when the number of processes increases.

Mock Grab Ticket

# Create Db.txt File # {"Count": 1}from multiprocessing import     process,lockimport json,os,time,randomdef search ():    With open (' Db.txt ', encoding= ' utf-8 ') as f:        dic = Json.load (f)        print ('%s remaining votes%s '% (Os.getpid (), dic[' count ')) Def get ():    with open (' Db.txt ', encoding= ' Utf-8 ') as Read_f:        dic = Json.load (read_f)        if dic[' count '] > 0:< c7/>dic[' count ']-= 1            time.sleep (Random.randint (1,3))  # Analog hand speed and speed with            open (' Db.txt ', ' W ', encoding= ' Utf-8 ') as Write_f:                json.dump (dic,write_f)                print ('%s Snatch ticket succeeded '%os.getpid ()) def task (mutex):    search ()    Mutex.acquire ()    get ()    mutex.release () if __name__ = = ' __main__ ':    mutex = Lock () for    I in Range:        p = Process (target=task,args= (mutex))        P.start ()

Signal Volume

mutexes allow only one thread to change data at the same time, while Semaphore allows a certain number of threads to change data, such as the toilet has 3 pits, the maximum allows only 3 people to the toilet, the back of the people can only wait inside someone out to go in, if the specified signal volume is 3, then a person to obtain a lock, Count plus 1,
When the Count equals 3 o'clock, the person behind needs to wait. Once released, someone can get a lock, the semaphore is similar to the concept of the process pool, but to differentiate, the semaphore involves the concept of lock-in from multiprocessing import Process,semaphoreimport time,randomdef GO_WC (sem,user):    sem.acquire ()    print ('%s ' takes up a Manger '%user)    time.sleep (Random.randint (0,3)) #模拟每个人拉屎速度不一样 , 0 means some people squat and get up.    sem.release () if __name__ = = ' __main__ ':    sem=semaphore (5)    p_l=[] for I in    Range (13):        p=process (target=go_wc,args= (SEM, ' user%s '%i,))        P.start ()        p_l.append (p) for    I in p_l:        I.join ()    print (' ============ ')

Process queue and Thread queue

From multiprocessing import Queue  # process queue import Queue  # thread Queue q = Queue (3) q.put ({' A ': 1}) Q.put (' xxxx ') q.put (3) Print (Q.get ()) print (Q.get ()) print (Q.get ()) # queue q = queue. Queue (3) q.put ({' A ': 1}) Q.put (' xxxx ') q.put (3) print (Q.get ()) print (Q.get ()) print (Q.get ()) # priority queue, the smaller the number, the higher the priority q = queue. Priorityqueue (3) q.put ((10,{' a ': 1}) Q.put (( -1, ' xxxx ')) Q.put ((0,3)) print (Q.get ()) print (Q.get ()) print (Q.get ()) # Stack q = queue. Lifoqueue (3) q.put ({' A ': 1}) Q.put (' xxxx ') q.put (3) print (Q.get ()) print (Q.get ()) print (Q.get ())

Producer Consumer Model

The producer-consumer model solves the problem of strong coupling between producers and consumers through a container. Producers and consumers do not communicate with each other directly, and through the blocking queue to communicate, so producers do not have to wait for consumer processing after the production of data, directly to the blocking queue, consumers do not find producers to data, but directly from the blocking queue, the blocking queue is equivalent to a buffer, Balance the processing power of producers and consumers.

From multiprocessing import Queue,processimport time,random,osdef producer (q): For    i in range:        res = ' bun%s '% I        time.sleep (0.5)        q.put (res)        print ('%s produced%s '% (Os.getpid (), RES)) def consumer (q): While    True:        res = Q.get ()        If res is none:break        print ('%s eats%s '% (Os.getpid (), res))        Time.sleep (Random.randint (2,3)) if __ name__ = = ' __main__ ':    q = Queue ()    p = Process (target=producer,args= (q,))    C = Process (Target=consumer, args= (q,))    P.start ()    C.start ()    p.join ()    q.put (None)    print (' master ')

Daemon, mock grab ticket example, mutex, semaphore, queue summary