Python thread queue

Thread queue describes the type of queue:

• • FIFO:

Queue.queue (maxsize=0)

FIFO is the first in. The queue provides a basic FIFO container, which is simple to use, and MaxSize is an integer that indicates the upper bound of the number of data that can be stored in the queue. Once the upper limit is reached, the insertion causes blocking until the data in the queue is consumed. If MaxSize is less than or equal to 0, there is no limit to the queue size.

Example:

Import Queueq = queue. Queue () q.put (1) q.put (2) q.put (3) print (Q.get ()) print (Q.get ()) print (Q.get ())

Output:

123

• • Lifo

Queue.lifoqueue (maxsize=0)

LIFO is last in first out, LIFO. Similar to the stack, the use is also very simple, maxsize usage ibid.

Example:

Import Queueq = queue. Lifoqueue () q.put (1) q.put (2) q.put (3) print (Q.get ()) print (Q.get ()) print (Q.get ())

Output:

321

• • Priority

Class Queue.priorityqueue (maxsize=0)

Constructs a priority queue. MaxSize usage Ibid.

Example:

Import Queueq =queue. Priorityqueue (3) Q.put ((2, ' Aaron ')) Q.put (( -1, ' Jim ')) Q.put ((5, ' Jack ')) print (Q.get ()) print (Q.get ())

Output:

( -1, ' Jim ') (2, ' Aaron ') (5, ' Jack ')

Basic methods:

Queue.queue (maxsize=0) FIFO, if maxsize less than 1 means that the queue length is infinite
Queue.lifoqueue (maxsize=0) LIFO, if maxsize less than 1 means that the queue length is infinite
Queue.qsize () returns the size of the queue
Queue.empty () returns True if the queue is empty, and vice versa false
Queue.full () returns True if the queue is full, otherwise false
Queue.get ([block[, timeout]]) Read queue, timeout wait time
Queue.put (item, [block[, timeout]]) write queue, timeout wait time
Queue.queue.clear () Empty queue

Other:

Task_done ()

means that one of the previous missions has been completed. Called by the consumer thread of the queue. Each get () call gets a task, and the next Task_done () call tells the queue that the task has been processed.

If the current join () is blocking, it resumes execution when all the tasks in the queue have been processed (that is, every task queued by a put () call has a corresponding task_done () call).

Join ()

Blocks the calling thread until all tasks in the queue are processed.

As long as there is data queued, the number of unfinished tasks increases. The number of unfinished tasks is reduced when the consumer thread calls Task_done () (meaning that there is a consumer getting the task and completing the task). When the number of unfinished tasks drops to 0,join () is unblocked.

Producer Consumer Model

Using producer and consumer patterns in concurrent programming can solve most concurrency problems. This mode improves the overall processing speed of the program by balancing the productivity of the production line and the consuming thread.

Why use producer and consumer models

In the world of threads, the producer is the thread of production data, and the consumer is the thread of consumption data. In multithreaded development, producers have to wait for the consumer to continue producing data if the producer is processing fast and the consumer processing is slow. Similarly, consumers must wait for producers if their processing power is greater than that of producers. To solve this problem, the producer and consumer models were introduced.

What is the producer consumer model

The producer-consumer model solves the problem of strong coupling between producers and consumers through a container. Producers and consumers do not communicate with each other directly, and through the blocking queue to communicate, so producers do not have to wait for consumer processing after the production of data, directly to the blocking queue, consumers do not find producers to data, but directly from the blocking queue, the blocking queue is equivalent to a buffer, Balance the processing power of producers and consumers.

A simple example of producer consumers

Import Threadingimport Queue def producer ():    for I in range:        q.put ("Bone%s"% i)     print ("Start waiting for all bones to be taken away ...")    q.join () #等待所有任务完成    print ("All bones are finished ...")  def Consumer (n): While     q.qsize () >0:         print ("%s Fetch" %n  , Q.get ())        Q.task_done () #告知这个任务执行完了  q = queue. Queue () p = Threading. Thread (Target=producer,) p.start () c1 = consumer ("Li Chuang")

Attention:

• If there is no Q.task_done (), the thread queue is not notified, and the producer is blocked.
• If there is no Q.join (), the producer will not wait for the consumer, it will end itself.
• The Q.task_done () method and the Q.join () method exist to realize the synchronization mechanism between producer and consumer.

Import Time,randomimport Queue,threadingq = queue. Queue () def Producer (name):  count = 0 while  count <20:    time.sleep (Random.randrange (3))    Q.put ( Count)    print (' Producer%s has produced%s Baozi.. '% (name, count))    Count +=1def Consumer (name):  count = 0
   
    while count <20:    time.sleep (Random.randrange (4))    if not Q.empty ():        data = Q.get ()        print (data)        print (' \033[32;1mconsumer%s has eat%s baozi...\033[0m '% (name, data))    else:        print ("-----No Baozi Anymore----")    count +=1p1 = Threading. Thread (Target=producer, args= (' A ',)) C1 = Threading. Thread (Target=consumer, args= (' B ',)) P1.start () C1.start ()
   

This example, fully embodies the thread queue in the producer consumer model, is how to solve the problem of strong coupling between producers and consumers.

Thread Queue Usage Considerations 1. Blocking mode

Import Queueq = queue. Queue (Ten)  #创建一个队列       ... For I in range:               q.put (' A ')               time.sleep (0.5)

This is a very simple code (another two threads are also operating queue Q), I expect to write a ' a ' to the queue every 0.5 seconds, but always do not want: the interval is sometimes far more than 0.5 seconds. Originally, Queue.put () has block = True and timeou two parameters by default. When block = True, the write is blocked and the blocking time is determined by Timeou. When the queue q is filled (other threads), the code blocks until the other thread takes the data. The Queue.put () method, combined with the Block=false parameters, solves this hidden problem. Note, however, that a non-blocking write queue throws an exception Queue.full exception when the queue is full.

2. Unable to capture exception Queue.empty exception

While True:    ...     Try:         data = Q.get ()     except Queue.empty:          break

The intention is to use the queue as empty, exit the loop, but actually run, but into a dead loop. This problem is somewhat similar to the above: Queue.get () The default is blocking mode to read the data, the queue is empty, will not throw except Queue.empty, but into blocking until time-out. The parameter of the Get method is Block=false, and the problem is solved.

Python thread queue