Python Full Stack Development Basics "25th" deadlock, recursion lock, Semaphore, event event, thread queue

One, deadlock phenomenon and recursive lock

The process also has a deadlock.

The so-called deadlock: refers to two or two or more processes or threads in the course of execution, because of the contention for resources caused by a mutual waiting phenomenon, if there is no external force,

They will not be able to go forward. At this point the system is in a deadlock state or the system generates a deadlock, and these processes, which are always waiting for each other, are called deadlock processes.

Here's the deadlock.

#死锁现象死锁-------------------from  threading import Thread,lock,rlockimport Timemutexa = Lock () Mutexb = Lock () class MyThread (Thread):    def run (self):        self.f1 ()        self.f2 ()    def F1 (self):        mutexa.acquire ()        Print (' \033[33m%s get a lock '%self.name)        mutexb.acquire ()        print (' \033[45%s get B Lock '%self.name)        mutexb.release ()        mutexa.release ()    def f2 (self):        mutexb.acquire ()        print (' \033[33%s get B lock '% self.name)        Time.sleep (1)  #睡一秒就是为了保证A锁已经被别人那到了        mutexa.acquire ()        print (' \033[45m%s get B lock '% self.name)        Mutexa.release ()        mutexb.release () if __name__ = = ' __main__ ': for    i in range:        t = MyThread ()        T.start () #一开启就会去调用run方法

So how to solve the deadlock phenomenon?

Workaround, Recursive lock: In Python to support multiple requests for the same resource in the same thread, Python provides a reentrant lock rlock.

The Rlock internally maintains a lock and a counter variable, counter records the number of acquire, so that resources can be require multiple times.

Until all the acquire of a thread are release, the other threads can get the resources. In the example above, if you use Rlock instead of lock, a deadlock will not occur

# a thread to get the lock, counter plus 1, the line range again hit the lock situation,<br> counter continue to add 1, during which all other threads can only wait, waiting for the thread to release all locks, that is, counter down to 0

# Method of resolving deadlocks--------------recursive lock from  threading import thread,lock,rlockimport Timemutexb = Mutexa = Rlock () class MyThread (Thread):    def run (self):        self.f1 ()        self.f2 ()    def F1 (self):        mutexa.acquire ()        print (' \033[ 33m%s get a lock '%self.name)        mutexb.acquire ()        print (' \033[45%s get B Lock '%self.name)        mutexb.release ()        Mutexa.release ()    def f2 (self):        mutexb.acquire ()        print (' \033[33%s get B lock '% self.name)        Time.sleep (1)  #睡一秒就是为了保证A锁已经被别人拿到了        mutexa.acquire ()        print (' \033[45m%s get B lock '% self.name)        mutexa.release ()        Mutexb.release () if __name__ = = ' __main__ ': for    i in range:        t = MyThread ()        T.start () #一开启就会去调用run方法

Second, the semaphore semaphore (actually also a lock)

Semaphore managing a built-in counter

The semaphore looks similar to the process pool, but it is a completely different concept.

Process pools: Pool (4), the maximum can only produce four processes, and from beginning to end are only these four processes, will not produce new.

Semaphore: The semaphore is generated by a bunch of processes/threads that generate multiple tasks to grab that lock.

# Semaphore Example from threading import Thread,semaphore,currentthreadimport time,randomsm = Semaphore (5) #运行的时候有5个人def task ():    sm.acquire ()    print (' \033[42m%s toilet '%currentthread (). GetName ())    Time.sleep (Random.randint (1,3))    print (' \033[31m%s on the toilet Gone '%currentthread (). GetName ())    sm.release () if __name__ = = ' __main__ ': for    i in Range:  #开了10个线程, all 20 people have to go to the bathroom.        t = Thread (target=task)        T.start ()

　　

Third, the Event

A key feature of a thread is that each thread is run independently and the state is unpredictable. Thread synchronization problems can become tricky if other threads in the program need to determine the state of a thread to decide what to do next. To solve these problems, we need to use the event object in the threading library. The object contains a signal flag that can be set by the thread, which allows the thread to wait for certain events to occur. In the initial case, the signal flag in the event object is set to False. If the thread waits for an event object, and the flag of the event object is false, then the threads will be blocked until the flag is true. A thread if the signal flag of an event object is set to true, it will wake up all the threads waiting for the event object. If a thread waits for an event object that is already set to true, it ignores the event and continues execution

Rom Threading Import Eventevent.isset () #返回event的状态值Event. Wait () #如果 Event.isset () ==false will block the thread; Event.set () # Set the status value of Event to true, all blocking pool threads are activated into a ready state, waiting for the operating system to dispatch; Event.clear () #恢复

Example 1: There are multiple worker threads trying to link MySQL, we want to make sure that the MySQL service is OK before linking to the MySQL server, if the connection is unsuccessful, try to reconnect. Then we can use threading. Event mechanism to coordinate the connection operations of individual worker threads

#首先定义两个函数, one is to connect the database # One is to detect the database from threading import Thread,event,currentthreadimport Timee = Event () def conn_mysql ():    ' link database '    count = 1    while not E.is_set ():  #当没有检测到时候        if Count >3: #如果尝试次数大于3, throw an exception            Raise Connectionerror (' too many attempts to link ')        print (' \033[45m%s%s attempts '% (CurrentThread (), count))        e.wait (timeout=1) # Wait for detection (inside the parameter is timeout 1 seconds)        count+=1    print (' \033[44m%s start link ... '% (CurrentThread (). GetName ())) def check_mysql ():    "Detection Database"    print (' \033[42m%s detection mysql ... '% (CurrentThread (). GetName ()))    Time.sleep (5)    E.set () if __name__ = = ' __main__ ': for    i in  range (3):  #三个去链接        t = Thread (target=conn_mysql)        T.start ()    t = Thread (target=check_mysql)    T.start ()

Example 2, example of a traffic light

From  threading Import Thread,event,currentthreadimport Timee = Event () def traffic_lights ():    "' Traffic light '    Time.sleep (5)    E.set () def car ():    "Car"    print (' \033[42m%s, etc green light \033[0m '%currentthread (). GetName ())    e.wait ()    print (' \033[44m%s car start pass '% CurrentThread (). GetName ()) If __name__ = = ' __main__ ': for    I in range (10 ):        t = Thread (target=car)  #10辆车        t.start ()    traffic_thread = Thread (target=traffic_lights)  # A traffic light    Traffic_thread.start ()

Four, timers (timer)

Specify n seconds after an action is performed

 from Import Timer def func (N):     Print ('hello,world'= Timer (3,func,args= (123,))  # wait three seconds to execute the Func function, because the Func function has parameters, then pass a parameter in T.start ()

Five, Thread queue

Queue queues: Use import queue, same usage as process queue

queue.Queue(maxsize=0) #先进先出

# 1. Queue-----------Import Queueq = queue. Queue (3) #先进先出q. Put (' first ') q.put (' second ') Q.put (' third ') print (Q.get ()) print (Q.get ())

queue.LifoQueue(maxsize=0) #先进后出

# 2. Stack----------q = queue. Lifoqueue () #先进后出 (or LIFO) Q.put (' first ') q.put (' second ') Q.put (' third ') q.put (' for ') print (Q.get ()) print (Q.get ()) Print (Q.get ())

queue.PriorityQueue(maxsize=0) #存储数据时可设置优先级的队列

# 3.put into a tuple, the first element of the tuple is the priority----------------(usually also a number, or it can also be a comparison between non-numbers) the smaller the number, the higher the priority ' q = Queue '. Priorityqueue ((Q.put, ' a ')) Q.put ((b, ' a '))  #先出来的是b, the smaller the number, the higher the priority of the Q.put (() ((c)) print (Q.get ()) ) Print (Q.get ())

Six, multi-threaded performance test

1. Multi-core is more than one CPU
(1) The more CPU, the higher the performance of the calculation
(2) If the program is IO operation (multicore and single core is the same), no more CPU is meaningless.
2. Implementing concurrency
The first type: one process, multiple threads open
The second type: Open multiple processes
3. Multi-process:
   Pros: Can take advantage of multi-core
   Cons: Large Overhead
4. Multithreading
   Pros: Less overhead
   Cons: No multi-core can be used
5 multi-process and multi-process scenarios
   1. Compute-intensive: that is, more computation, less IO
     If it is computationally intensive, use multiple processes (e.g. financial analysis, etc.)
   2.IO Intensive: More IO, less computation
     If it is IO-intensive, use multithreading (generally encountered IO-intensive)
The following example exercises:

# compute intensive to open multi-process from multiprocessing import processfrom threading Import    Threadimport timedef Work (): res = 0 for i in range (10000000): res+=iif __name__ = = ' __main__ ': l = []  Start = Time.time () for I in range (4): P = Process (target=work) #1.9371106624603271 #可以利用多核 (i.e. multiple CPUs) # p = Thread (target=work) #3.0401737689971924 l.append (P) p.start () for P in L:p.join () stop = Time.time () print ('%s '% (Stop-start)) 

# I/O intensive to turn on multithreading from multiprocessing import processfrom threading Import Threadimport timedef work ():    time.sleep (3) if __name__ = = ' __main__ ':    l = []    start = Time.time () for    I in range:        # p = Process (target=work)  #34 .9549994468689   #因为开了好多进程, it costs a lot of time, and it takes longer        p = Thread (target=work) #2.2151265144348145  #当开了多个线程的时候, It costs less and spends less time        l.append (p)        P.start () for    i in L:        i.join ()    stop = Time.time ()    print ('%s '% (Stop-start))

　　

Python Full Stack Development Basics "25th" deadlock, recursion lock, Semaphore, event event, thread queue