Python Development Threads: Deadlocks and Recursive locks & semaphores & timers & Thread queue& Events evevt

Tag: off cannot counter red FIFO seconds after tween priority

One deadlock phenomenon with recursive lock

The process also has a deadlock with a recursive lock, which in the process has forgotten to say, put it all here to say the amount

The so-called deadlock: refers to two or two or more processes or threads in the execution process, because of the contention for resources caused by a mutual waiting phenomenon, if there is no external force, they will not be able to proceed. At this point the system is in a deadlock state or the system produces a deadlock, the process that is always waiting for each other is called a deadlock process, as follows: Deadlock

From threading Import Thread,lockimport Timemutexa=lock () Mutexb=lock () class MyThread (Thread):    def run (self):        self.func1 ()        Self.func2 ()    def func1 (self):        mutexa.acquire ()        print (' \033[41m%s get a lock \033[0m '% Self.name)        mutexb.acquire ()        print (' \033[42m%s get B lock \033[0m '%self.name)        mutexb.release ()        Mutexa.release ()    def func2 (self):        mutexb.acquire ()        print (' \033[43m%s get B lock \033[0m '%self.name ')        Time.sleep (2)        mutexa.acquire ()        print (' \033[44m%s get a lock \033[0m '%self.name)        mutexa.release ()        mutexb.release () if __name__ = = ' __main__ ': for    i in range:        t=mythread ()        T.start () " Thread-1 get a lock Thread-1 get B lock Thread-1 get B lock Thread-2 get a lock and then stuck, dead lock "

View Code

Workaround, recursive lock, Python provides a reentrant lock Rlock in Python in order to support multiple requests for the same resource in the same thread.

The Rlock internally maintains a lock and a counter variable, counter records the number of acquire, so that resources can be require multiple times. Until all the acquire of a thread are release, the other threads can get the resources. In the example above, if you use Rlock instead of lock, a deadlock will not occur:

Mutexa=mutexb=threading. Rlock () #一个线程拿到锁, counter plus 1, the line range again in the case of lock, then counter continue to add 1, during which all other threads can only wait, waiting for the thread to release all locks, that is, counter down to 0

Two-signal volume semaphore

Same as the process

Semaphore manages a built-in counter,
Built-in counter whenever acquire () is called-1;
Built-in counter +1 when call Release ();
The counter cannot be less than 0, and when the counter is 0 o'clock, acquire () blocks the thread until another thread calls release ().

Example: (at the same time only 5 threads can get semaphore, that is, you can limit the maximum number of connections to 5):

From threading import Thread,semaphoreimport threadingimport time# def func (): #     if Sm.acquire (): #         Print ( Threading.currentthread (). GetName () + ' get semaphore ') #         Time.sleep (2) #         sm.release () def func ():    Sm.acquire ()    print ('%s get SM '%threading.current_thread (). GetName ())    Time.sleep (3)    sm.release () if __ name__ = = ' __main__ ':    sm=semaphore (5) for    I in range:        t=thread (Target=func)        T.start ()

View Code

Unlike the process pool, which is completely different from the concept of process pool (4), the maximum can only generate 4 processes, and from beginning to end it is just these four processes that do not produce new, and semaphores generate a bunch of threads/processes

Three Event events

Same as the process

A key feature of a thread is that each thread is run independently and the state is unpredictable. Thread synchronization problems can become tricky if other threads in the program need to determine the state of a thread to decide what to do next. To solve these problems, we need to use the event object in the threading library. The object contains a signal flag that can be set by the thread, which allows the thread to wait for certain events to occur. In the initial case, the signal flag in the event object is set to False. If the thread waits for an event object, and the flag of the event object is false, then the threads will be blocked until the flag is true. A thread if the signal flag of an event object is set to true, it will wake up all the threads waiting for the event object. If a thread waits for an event object that is already set to true, it ignores the event and continues execution

Event.isset (): Returns the status value of the event, event.wait (): If Event.isset () ==false will block the thread; Event.set (): Sets the status value of event to True, All blocking pool threads are activated into a ready state, waiting for the operating system to dispatch; Event.clear (): The status value of recovery event is false.

For example, there are multiple worker threads trying to link MySQL, and we want to make sure that the MySQL service is working properly before linking to the MySQL server, and if the connection is unsuccessful, try reconnecting. Then we can use threading. Event mechanism to coordinate the connection operations of individual worker threads

From threading import Thread,eventimport threadingimport time,randomdef conn_mysql ():    count=1 and not    Event.is_set ():        If Count > 3:            raise Timeouterror (' link timeout ')        print (' <%s>%s attempts link '% ( Threading.current_thread (). GetName (), count)        event.wait (0.5)        count+=1    print (' <%s> link success '% Threading.current_thread (). GetName ()) def check_mysql ():    print (' \033[45m[%s] checking mysql\033[0m '% Threading.current_thread (). GetName ())    Time.sleep (Random.randint (2,4))    Event.set () if __name__ = = ' __main_ _ ':    event=event ()    conn1=thread (target=conn_mysql)    conn2=thread (target=conn_mysql)    check= Thread (Target=check_mysql)    Conn1.start ()    Conn2.start ()    Check.start ()

View CodeFour timers

Timer, specifying n seconds after an action is performed

From threading Import timer  def hello ():    print (' Hello, world ') t = Timer (1, hello) t.start ()  # after 1 seconds, "Hello, World" would be printed

Five thread queue

Queue queues: Use import queue, same usage as process queue

Queue is especially useful in threaded programming when information must be exchanged safely between multiple threads.

queue. class Queue (maxsize=0) #先进先出

Import Queueq=queue. Queue () q.put (' first ') q.put (' second ') Q.put (' third ') print (Q.get ()) (Q.get ()) print (Q.get ()) "Results (FIFO): Firstsecondthird ""

View Code

queue. class LifoQueue (maxsize=0) #last in fisrt out

Import Queueq=queue. Lifoqueue () q.put (' first ') q.put (' second ') Q.put (' third ') print (Q.get ()) print (Q.get ()) print (Q.get ()) "Results (LIFO): Thirdsecondfirst ""

View Code

queue. class PriorityQueue (maxsize=0) #存储数据时可设置优先级的队列

Import Queueq=queue. Priorityqueue () #put进入一个元组, the first element of a tuple is a priority (usually a number, or a comparison between non-numbers), and the smaller the number the higher the Priority Q.put ((A (), ' a ')) Q.put (((a), ' B ')) Q.put (((), ' C ') print (Q.get ()) print (Q.get ()) print (Q.get ()) "Results (the smaller the number, the higher the priority, the priority queue):(, ' B ') (+, ' a ') (+, ' C ')

View Code

其他

Constructor for a priority queue. MaxSize is a integer that sets the upperbound limit on the number of items the can being placed in the queue. Insertion would block once this size have been reached, until queue items are consumed. If MaxSize is less than or equal to zero, the queue size is infinite. The lowest valued entries is retrieved first (the lowest valued entry was the one returned by sorted (list (entries)) [0]). A typical pattern for entries are a tuple in the form: (Priority_number, data). Exception queue. Emptyexception raised when Non-blocking get () (or get_nowait ()) are called on a queue object which is empty.exception queue . Fullexception raised when Non-blocking put (or put_nowait ()) are called on a Queue object which are full. Queue.qsize () Queue.empty () #return True if Empty queue.full () # return True if full Queue.put (item, Block=true, Timeout=n One) Put item into the queue. If optional args block is true and timeout are None (the default), block if necessary until a free slots is AVAilable. If timeout is a positive number, it blocks at most timeout seconds and raises the full exception if no free slots was avail Able within that time. Otherwise (block is false), put an item on the queue if a free slot is immediately available, else raise the full exceptio N (timeout is ignored in). Queue.put_nowait (item) equivalent to put (item, False). Queue.get (Block=true, Timeout=none) Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available.  If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available Within that time. Otherwise (block is false), return a item if one is immediately available, else raise the Empty exception (timeout is IGN ORed in the case). Queue.get_nowait () equivalent to get (False). The methods is offered to the support tracking whether enqueued tasks has been fully processed by daemon consumer threads.Queue.task_done () indicate that a formerly enqueued task was complete. Used by queue consumer threads. For each get () used to fetch a task, a subsequent call to Task_done () tells the queue, the processing on the task is C Omplete. If a Join () is currently blocking, it would resume when all items has been processed (meaning that a task_done () call was Received for every item this had been put () into the queue). Raises a valueerror if called more times than there were items placed in the queue. Queue.join () block until the queue is consumed

View Code

Python Development Threads: Deadlocks and Recursive locks & semaphores & timers & Thread queue& Events evevt