Python Development Foundation---Event object, queue, and multi-process Foundation

Event Object

Used for inter-thread communication, where a thread in a program needs to determine its next operation by judging the state of a thread, using the event object

The event object defaults to False (Flase), which is the execution of a thread that encounters an event object waiting to be blocked.

Example 1: Communication between the main thread and the child threads, the code simulates the connection server

1 Import Threading 2 Import time 3 event=threading. Event () 4  5 def foo (): 6     print (' Wait server ... ') 7     event.wait ()    #括号里可以带数字执行, number indicates the number of seconds to wait, no number indicates blocking status 8     print (' Connect to server ') 9 t=threading. Thread (target=foo,args= ())  #子线程执行foo函数11 T.start () time.sleep (3) print (' Start server successful ') 14 Time.sleep (3) event.set ()     #默认为False, set represents true at once, so the Foo function of the strand thread continues to be unblocked

Example 2: Communication between child threads and child threads

1 Import Threading 2 Import time 3 event=threading. Event () 4  5 def foo (): 6     print (' Wait server ... ') 7     event.wait ()    #括号里可以带数字执行, number indicates the number of seconds to wait, no number indicates blocking status 8     print (' Connect to server ') 9 def start ():     time.sleep (3)     print (' Start server successful ')     Time.sleep (3)     event.set ()     #默认为False, set represents true at one time, so the Foo function of the subnet thread is unblocked and continues to execute the t=threading. Thread (target=foo,args= ())  #子线程执行foo函数15 T.start () t2=threading. Thread (target=start,args= ())  #子线程执行start函数17 T2.start ()

Example 3: Multithreading blocking

1 Import Threading 2 Import time 3  4 event=threading. Event () 5 def foo (): 6     while not Event.is_set ():   #返回event的状态值, with Isset 7         print ("Wait Server ...") 8         Event.wait (2)   #等待2秒, if the status is False, print a prompt to continue waiting 9     print ("Connect to Server") Ten for     I in range (5):  # 5 sub-threads wait at the same time for a     t=threading. Thread (target=foo,args= ())     T.start () print ("Start server Successful") Time.sleep () Event.set ()   # Setting the flag bit to True,event.clear () is the status value of the reply event is false

Queue queues

A queue is a data structure that is stored in a way that is similar to a list, but takes data in a different way from a list.

There are three ways to queue data:

1, first-out (FIFO), that is, which data first deposit, take data when the first to take which data, with the life of the queue to buy things

2, Advanced post-out (LIFO), the same stack, that is, which data was last deposited, take the data when the first take, with the life of the pistol cartridges, bullets last put into the first

3, priority queue, that is, when the data is added to a priority, take the data when the highest priority to remove

Code implementation

FIFO: Put deposit and get out

1 Import Queue 2 import threading 3 Import time 4 q=queue. Queue (5) #加数字限制队列的长度, can deposit up to 5 data, can be removed to continue to deposit 5 def put (): 6 for     i in range:    #顺序存入数字0到99 7         q.put (i) 8         Time.sleep (1)   #延迟存入数字, when there is no data in the queue, the Get function will block when the data is fetched, until the data is stored and then released from the blocking state to remove the new data 9 def get (): Ten for     I in range (100):    #从第一个数字0开始取 until 9911         print (Q.get ()) t1=threading. Thread (target=put,args= ()) T1.start () t2=threading. Thread (target=get,args= ()) T2.start ()

FIFO: Join blocking and Task_done signals

1 Import Queue 2 import threading 3 Import time 4 q=queue. Queue (5) #加数字限制长度 5 def put (): 6     for I in range: 7         q.put (i) 8     q.join ()    #阻塞进程 until all tasks are completed, how many times data is taken Task_ Done how many times, otherwise the last OK cannot print 9     print (' OK ') def get (): Three for I in     Range:         print (Q.get ())         Q.task_ Done ()   #必须每取走一个数据, send a signal to Join15     # q.task_done ()   #放在这没用, because join is actually a counter, put the number of data,                       #计数器就是多少, Once per Task_done, the counter is reduced by 1 to 0 before continuing to perform the t1=threading. Thread (target=put,args= ()) T1.start () t2=threading. Thread (target=get,args= ()) T2.start ()

Advanced Post-out:

1 Import Queue 2 import threading 3 Import time 4  5 q=queue. Lifoqueue () 6 def put (): 7 for     I in range: 8         q.put (i) 9     q.join ()     print (' OK ') one def get ():     For I in range: +         print (Q.get ())         Q.task_done () t1=threading. Thread (target=put,args= ()) T1.start () t2=threading. Thread (target=get,args= ()) T2.start ()

By priority: Whether it is a number, letters, lists, tuples, etc. (dictionaries, sets are not measured), using priority to save data to the data, the data in the queue must be the same type, are based on the actual data in the order of the ASCII table priority matching, Chinese characters are in accordance with the Unicode table (pro-Test)

List

1 Import Queue 2 q=queue.  Priorityqueue () 3 Q.put ([1, ' AAA ']) 4 q.put ([1, ' Ace ']) 5 q.put ([4,333]) 6 q.put ([3, ' AfD ']) 7 q.put ([5, ' 4ASDG ']) 8 #1是级别最高的, 9 While not Q.empty (): #不为空时候执行10     print (Q.get ())

Meta-group

1 Import queue2 Q=queue. Priorityqueue () 3 Q.put ((1, ' AAA ')) 4 Q.put ((1, ' Ace ')) 5 q.put ((4,333)) 6 Q.put ((3, ' AfD ')) 7 Q.put ((5, ' 4ASDG ')) 8 and not Q.empty (): #不为空时候执行9     print (Q.get ())

Chinese characters

1 Import queue2 Q=queue. Priorityqueue () 3 q.put (' I ') 4 q.put (' You ') 5 q.put (' he ') 6 q.put (' she ') 7 q.put (' Ta ') 8 while Not Q.empty (): 9     print (Q.get ())

Producer and Consumer models

In the world of threads, the producer is the thread of production data, and the consumer is the thread of consumption data. In multithreaded development, producers have to wait for the consumer to continue producing data if the producer is processing fast and the consumer processing is slow. Similarly, consumers must wait for producers if their processing power is greater than that of producers. To solve this problem, the producer and consumer models were introduced.

The producer-consumer model solves the problem of strong coupling between producers and consumers through a container. Producers and consumers do not communicate with each other directly, and through the blocking queue to communicate, so producers do not have to wait for consumer processing after the production of data, directly to the blocking queue, consumers do not find producers to data, but directly from the blocking queue, the blocking queue is equivalent to a buffer, Balance the processing power of producers and consumers.

This is like, in the restaurant, cook cooked, do not need to communicate directly with customers, but to the front desk, and customers do not need to go to the food chef, directly to the front desk to pick up, which is also a coupling process.

 1 Import time,random 2 import queue,threading 3 4 q = queue. Queue () 5 6 def Producer (name): 7 count = 0 8 while count <10:9 print ("Making ...") Time.sleep (Rand     Om.randrange (3)) one q.put (count) of print (' Producer%s has produced%s Baozi: '% (name, count)) Count +=114     #q. Task_done () #q. Join () print ("OK ..."), Def Consumer (name): Count = 019 while Count <10:20 Time.sleep (Random.randrange (4)) if not Q.empty (): All data = Q.get () #q. Task_done () #q. Jo          In () print (' \033[32;1mconsumer%s have eat%s baozi...\033[0m '% (name, data)) else:28 Print ("-----no Baozi anymore----") count +=130 to P1 = Threading. Thread (Target=producer, args= (' A ',)) is C1 = Threading. Thread (Target=consumer, args= (' B ',)) # C2 = Threading. Thread (Target=consumer, args= (' C ',)) # C3 = Threading. Thread (Target=consumer, args= (' D ',)) P1.start () C1.start () Notoginseng # C2.start () 38 #C3.start () 

Multi-Process Foundation

Because of the Gil's existence, multithreading in Python is not really multi-threading, and if you want to fully use the resources of multicore CPUs, most of the situations in Python need to use multi-process

Multi-Process Advantages: Multi-core can be used to achieve parallel operation

Multi-process Disadvantage: Switching overhead is too high, inter-process communication is difficult

Multiprocessing Module

The multiprocessing package is a multi-process Management Pack in Python. With threading. Like thread, it can take advantage of multiprocessing. Processes object to create a process. The process can run functions written inside the Python program. The process object is used in the same way as the thread object, and there is a method for start (), run (), join (). There are also lock/event/semaphore/condition classes in the multiprocessing package (these objects can be passed to each process as multi-threaded, through parameters) to synchronize the process, using the same class as the threading package. So, a large part of multiprocessing and threading use the same set of APIs, but in a multi-process environment.

Compute-Intensive Serial computing: Calculation results about 25 seconds

1 Import Time 2  3 def foo (n):    #计算0到1亿的和 4     ret=0 5 for     i in range (n): 6         ret+=i 7     print (ret) 8  9 def bar (n):    #计算1到10万的乘积10     ret=111 for     i in range (1,n):         ret*=i13     Print (ret) if __name__ = = ' _ _main__ ':     s=time.time () 100000     foo (100000000) +     Bar () 18     

Compute-intensive multi-process computing: Calculation results of about 13 seconds

1 Import multiprocessing 2 import time 3  4 def foo (n): 5     ret=0 6 for     i in range (n): 7         ret+=i 8     print (r ET) 9 def bar (n): ret=112 for     i in Range (1,n):         ret*=i14     Print (ret) if __name__ = = ' __main__ ': 1 7     s=time.time ()     p1 = multiprocessing. Process (target=foo,args= (100000000)) #创建子进程, Target: The method to execute; Name: Process name (optional); Args/kwargs: The parameter to pass in the method.     P1.start () #同样调用的是类的run方法20     p2 = multiprocessing. Process (target=bar,args= (100000,)) #创建子进程21     P2.start ()     p1.join ()     p2.join () 24     

Inheriting class usages

1 from multiprocessing Import Process 2 import time 3  4 class myprocess (Process): 5     def __init__ (self): 6         Supe R (myprocess, self). __init__ () 7         # self.name = name 8  9     def run (self):         print (' Hello ', self.name, Time.ctime ())         Time.sleep (1) if __name__ = = ' __main__ ':     p_list=[]15 for I in     Range (3):         p = Myprocess ()         P.start ()         p_list.append (P),     p in p_list:21         p.join (),     print (' End ')

Method Example

1 from multiprocessing Import Process 2 import OS 3 import time 4  5 def info (name): 6     Print ("Name:", name) 7     PR Int (' Parent process: ', Os.getppid ())    #获取父进程的id号 8     print (' Process ID: ', os.getpid ())    #获取当前进程pid 9     Print ("------------------")     Time.sleep (5) One-if __name__ = = ' __main__ ': +     info (' main process ')    # The first time to get the process of the IDE tool and the process of the code file is     p1 = Processes (Target=info, args= (' Alvin ',))    #该代码文件的进程和p1的进程14     P1.start ()     P1.join ()

Methods for object instances

Instance method: Is_alive (): Returns whether the process is running.　　Join ([timeout]): Blocks the process of the current context until the process calling this method terminates or arrives at the specified timeout (optional parameter).　　Start (): The process is ready to wait for the CPU to dispatch run (): Strat () calls the Run method, and if the instance process does not have an incoming target set, this star executes the T default run () method.　　Terminate (): Stops worker process properties immediately regardless of whether the task is completed: daemon: Same as thread's Setdeamon function name: process name. PID: Process number.

Python Development Foundation---Event object, queue, and multi-process Foundation