Python Multithreading-thread Threading queue-Simple Learning

Python Multithreading-thread Threading queue-Simple Learning

In the actual work process, there will be a need to do something concurrency, such as a machine to measure the network connectivity of thousands of machines, if you single-threaded one to test the words, will spend a lot of things, not real-time, and can not be immediately aware of the situation of the network at the time of change, then multithreading is a good choice. Python has encapsulated multi-line libraries thread and threading.

thread: Comparing the underlying module
Threading: Higher-level Threading interface

PS: Recommended use of threading module
-High-level threading modules are more advanced and support for threading is more complete
-Low-level thread module synchronization primitives are rare
-The thread module has no control over when the threads end, and when the main thread ends, all threads will force the end

Thread

module functions

Start_new_thread (function, Args,kwargs=none): Generates a new thread, args is the parameter of the function, without writing (), Kwargs is used to call this function
Allocate_lock (): Allocation lock, LockType type
Exit (): Let the thread exit

Operation of the LockType

Acquire (Wait=none): attempt to acquire lock
Locked (): Gets the lock return True, no return false
Release (): Releasing the lock

Demo1

$ cat t1.pyimport threadfrom time import sleepdef a():    print "a start"    sleep(2)    print "a end"def b():    print "b start"    sleep(2)    print "b end"def main():    thread.start_new_thread(a,())    thread.start_new_thread(b,())    print "all done"if __name__ == "__main__":    main()$ python t1.pyall doneb starta start

It turns out that the results of each run are likely to be different, but there will never be "a End" and "B End". This is why there is no code written to let the main thread stop and wait until all the child threads have finished, so the main threads print "all done" shut down the A and b two threads after execution. How to do it, you can add a sleep here wait for the child process to finish executing before exiting.

Demo2:thread-Multi-threaded demo by sleep

$ cat t2.pyimport threadfrom time import sleepdef a():    print "a start"    sleep(2)    print "a end"def b():    print "b start"    sleep(2)    print "b end"def main():    thread.start_new_thread(a,())    thread.start_new_thread(b,())    sleep (4)       ----防止主进程过早退出，加sleep等待子进程执行完毕后再推出    print "all done"if __name__ == "__main__":    main()$ python t1.py b starta starta endb endall done

But suppose we don't know what happens when the child process executes, that's where the lock comes in. Because it is more reasonable to use a lock than to use the sleep () function. As shown below:

Demo3:thread--Multithreaded Demo by lock

The implementation is: the main thread initializes two locks, respectively to two functions, two functions to release the lock after executing their own code, the main thread has been polling this lock is not released, if released on the exit.

def a(lock, nsec):    print "a starting at :", ctime()    sleep(nsec)    lock.release()    -- 执行完之后释放锁    print "a end", ctime()def b(lock, nsec):    print "b starting at :", ctime()    sleep(nsec)    lock.release()    -- 执行完之后释放锁    print "b end", ctime()def main():    print "Demo Starting at:", ctime()    locks = []    # Initialize lock  -- 主线程先获取两个锁，占为己有    for i in range(2):        lock = thread.allocate_lock()        lock.acquire()        locks.append(lock)    # 每个进程分配一个锁    thread.start_new_thread(a, (locks[0],2))    thread.start_new_thread(b, (locks[1],4))    for i in range(2):   #一直在轮询，看锁有没有释放        while locks[i].locked(): pass     print "all done at:", ctime()

The final result is:

$ python thread_demo.py Demo Starting at: Fri Aug 29 22:03:01 2014a starting at : Fri Aug 29 22:03:01 2014b starting at : Fri Aug 29 22:03:01 2014a end Fri Aug 29 22:03:03 2014b end Fri Aug 29 22:03:05 2014all done at: Fri Aug 29 22:03:05 2014

It is not difficult to find that the synchronization mechanism of the thread library is more difficult to use, and everything needs to be handled by the master process. And there is no daemon, the whole world will become very quiet when the main process is retired. and the threading library provides a daemon for us. Here's a look at the simple usage of threading.

Threading

Threading provides the thread class, and also provides a number of very useful synchronization mechanisms. It is important to understand that the thread class is available, multi-threaded, that is, through multiple instances of the thread class. the main methods of the class are:

Start (): Starts the execution of the thread. In the thread library, there is no way to control the start of the thread
Join (Timeout=none): Waits for thread to end, somewhat similar to polling in Demo3
run (): Defines the functionality of the thread

It is more important to feel above, it will be used immediately. There are some other:

getName (): Gets the thread name
setName (name): Set thread name
isAlive (): Returns bool indicating whether the thread is running
Activecount (): Returns the number of threads in the run
CurrentThread (): Returns the current thread object
Enumerate (): Returns the list of currently active threads
Isdaemon (): returns the thread's daemon flag
Setdaemon (daemonic): Sets the daemon flag for a thread, typically called before the start () function
Settrace (func): Set trace functions for all threads
Setprofile (func): Sets the profile function for all threads

Demo4--Threading Demo

def loop(i, nsec):    print "thread %d starting at : %s" %(i, ctime())    sleep(nsec)    print "thread %d end at : %s" %(i, ctime())def main():    threads = []    loops = [2, 4]    # 实例化进程    for i in range(len(loops)):        t = threading.Thread(target = loop, args = (i, loops[i]))        threads.append(t)    for i in range(len(loops)):        threads[i].start()    for i in range(len(loops)):        threads[i].join()    print "all done"

The final result is:

thread 0 starting at : Sun Aug 31 13:31:28 2014thread 1 starting at : Sun Aug 31 13:31:28 2014thread 0 end at : Sun Aug 31 13:31:30 2014thread 1 end at : Sun Aug 31 13:31:32 2014all done

Visible threading can easily control the start of the thread, and wait for the end of each thread, and also do not have to set locks, release locks, which are encapsulated by the threading library, more advanced than the thread. In an actual operations project, multiple threads may be required to perform the same task, and a task pool is required. Each thread takes a task from the task pool, executes, then takes the task, and then executes until the task pool is empty, exiting the thread. Here you will find the Queue library to be described below.

Queue

The queue module can be used for multi-threaded communication, allowing each thread to share data, and producers to put the goods into the queue for consumers (threads) to use. In Python3, the queue module is named queue. the queue objects are:

Queue.queue (maxsize=0): creates a FIFO (first in first out)-queue object of size maxsize, this queue will be infinite if maxsize is not set.
Queue.lifoqueue (maxsize=0): Create a first-in, out-of-the-box object, the stack, and add queue.priorityqueue (maxsize=0) to the python2.6: queue with priority, Join in the python2.6

The methods of the queue object are:

qsize (): Returns the size of the queue
Empty (): NULL when returning to queue
Full (): Returns whether the queue is filled
put (item,block=0): put data into a queue object, block not 0 o'clock, wait until there are controls in the queue
Get (block=0):, as above, block is not 0 o'clock, will wait until there is data in the queue

Demo5--use of the queue demo :
Scenario: There are some integers in the queue, need to take the integer out, and the time to sleep the integer size, the following demo, is put 10 1, if the single-threaded word needs 10s

def work(q):    while True:        if q.empty():            return        else:            t = q.get()            time.sleep(t)def main():    q = Queue.Queue()   # 初始化一个Queue对象    for i in range(10): # 向Queue生产任务        q.put(1)    work(q)if __name__ == "__main__":    

The final result is:

time python threading_demo2.py real    0m10.085suser    0m0.060ssys 0m0.004s

A single thread would cost 10s.

Here's how to handle tasks in a queue with multithreading:

def work(q):    while True:        if q.empty():            return        else:            t = q.get()            time.sleep(t)def main():    q = Queue.Queue()    for i in range(10):        q.put(1)    thread_num = 10    threads = []    for i in range(thread_num):        t = threading.Thread(target = work, args = (q,)) # args需要输出的是一个元组，如果只有一个参数，后面加，表示元组，否则会报错        threads.append(t)    for i in range(thread_num):        threads[i].start()    for i in range(thread_num):        threads[i].join()

Look at the results below:

real    0m1.046suser    0m0.024ssys     0m0.020s

So for Python multi-threading, the main learning to use threading and queue, should be enough to deal with some problems in operations.

Python Multithreading-thread threading queue-Simple Learning