Python multithreading learning and python Multithreading

Python multithreading learning and python Multithreading

I. Usage of threads in Python:

In Python, there are two ways to use threads: functions or classes to wrap thread objects.

1. Functional Method: Call start_new_thread () function in the thread module to generate a new thread. For example:

import time  import thread  def timer(no, interval):      cnt = 0      while cnt<10:          print 'Thread:(%d) Time:%s\n'%(no, time.ctime())          time.sleep(interval)          cnt+=1      thread.exit_thread()          def test(): #Use thread.start_new_thread() to create 2 new threads      thread.start_new_thread(timer, (1,1))      thread.start_new_thread(timer, (2,2))     if __name__=='__main__':      test()  

The preceding example defines a thread function timer, which prints 10 time records and exits. The interval of each printing is determined by the interval parameter. Thread. the first parameter of start_new_thread (function, args [, kwargs]) is the thread function (timer method in this example), and the second parameter is the parameter passed to the thread function, it must be of the tuple type, and kwargs is an optional parameter.

The end of a thread can wait for the thread to terminate. You can also call the thread. exit () or thread. exit_thread () method in the thread function.

2. Create a subclass of threading. Thread to wrap a Thread object, as shown in the following example:

import threading  import time  class timer(threading.Thread): #The timer class is derived from the class threading.Thread      def __init__(self, num, interval):          threading.Thread.__init__(self)          self.thread_num = num          self.interval = interval          self.thread_stop = False         def run(self): #Overwrite run() method, put what you want the thread do here          while not self.thread_stop:              print 'Thread Object(%d), Time:%s\n' %(self.thread_num, time.ctime())              time.sleep(self.interval)      def stop(self):          self.thread_stop = True              def test():      thread1 = timer(1, 1)      thread2 = timer(2, 2)      thread1.start()      thread2.start()      time.sleep(10)      thread1.stop()      thread2.stop()      return     if __name__ == '__main__':      test()  

Personally, I prefer the second method, that is, to create my own Thread class. If necessary, rewrite the threading. Thread class method. Thread control can be customized by myself.

Use of threading. Thread class:

1. Call threading. Thread. _ init _ (self, name = threadname) in _ init _ of the Thread class)

Threadname indicates the thread name.

2. run (), which usually needs to be rewritten. Write the code to implement the required functions.

3, getName (), get the thread object name

4. setName (): Set the thread object name.

5, start (), start the thread

6. jion ([timeout]). Wait until another thread finishes running.

7. setDaemon (bool): Set whether the Sub-thread ends with the main thread and must be called before start. The default value is False.

8. isDaemon (): determines whether the thread ends with the main thread.

9, isAlive (), check whether the thread is running.

In addition, the threading module provides many methods and other classes to help us better use and manage threads. See http://www.python.org/doc/2.5.2/lib/module-threading.html.

 

 

Assume that both thread objects t1 and t2 need to increase num = 0 by 1, t1 and t2 each have to modify num 10 times, and the final result of num should be 20. However, due to multi-threaded access, the following situation may occur: When num = 0, t1 gets num = 0. At this time, the system schedules t1 to the "sleeping" state, converts t2 to the "running" state, and obtains num = 0 on page t2. Then t2 adds 1 to the obtained value and assigns it to num so that num = 1. Then the System Schedules t2 to "sleeping" and converts t1 to "running ". Thread t1 then assigns the value of 0 plus 1 to num. In this way, both t1 and t2 have completed 1 plus 1, but the result is still num = 1.

The preceding case describes one of the most common problems in multithreading: data sharing. When multiple threads want to modify a shared data, we need to synchronize data access.

1. Simple Synchronization

The simplest synchronization mechanism is "Lock ". The lock object is created by the threading. RLock class. The thread can use the lock's acquire () method to obtain the lock, so that the lock enters the "locked" state. Only one thread can obtain the lock at a time. If another thread tries to obtain the lock, it will be changed to the "blocked" state by the system until the thread that owns the lock calls the release () method of the lock to release the lock, in this way, the lock enters the "unlocked" state. A thread in the "blocked" State will receive a notification and have the right to obtain the lock. If multiple threads are in the "blocked" state, all threads will first release the "blocked" State, then the system selects a thread to obtain the lock, and other threads continue to silence ("blocked ").

The thread module and Lock Object in Python are low-level thread control tools provided by Python, which are easy to use. For example:

import thread  import time  mylock = thread.allocate_lock()  #Allocate a lock  num=0  #Shared resource    def add_num(name):      global num      while True:          mylock.acquire() #Get the lock           # Do something to the shared resource          print 'Thread %s locked! num=%s'%(name,str(num))          if num >= 5:              print 'Thread %s released! num=%s'%(name,str(num))              mylock.release()              thread.exit_thread()          num+=1          print 'Thread %s released! num=%s'%(name,str(num))          mylock.release()  #Release the lock.    def test():      thread.start_new_thread(add_num, ('A',))      thread.start_new_thread(add_num, ('B',))    if __name__== '__main__':      test()  

Python also provides an advanced thread control library based on the thread, which is the threading mentioned earlier. The threading module of Python is a module built on the thread module. Many attributes of the thread module are exposed in the threading module. In the thread module, python provides the user-level thread synchronization tool "Lock" object. In the threading module, python provides a variant of the Lock Object: RLock object. The RLock object maintains a Lock object, which is a reentrant object. For the Lock object, if a thread performs the acquire operation twice in a row, the second acquire will suspend the thread because there is no release after the first acquire operation. This will cause the Lock Object to never be release and cause a thread deadlock. The RLock object allows a thread to perform the acquire operation multiple times because it maintains the number of acquire threads through a counter variable. Each acquire operation must have a release operation. After all the release operations are completed, other threads can apply for the RLock object.

The following describes how to use the threading RLock object for synchronization.

import threading  mylock = threading.RLock()  num=0     class myThread(threading.Thread):      def __init__(self, name):          threading.Thread.__init__(self)          self.t_name = name                def run(self):          global num          while True:              mylock.acquire()              print '\nThread(%s) locked, Number: %d'%(self.t_name, num)              if num>=4:                  mylock.release()                  print '\nThread(%s) released, Number: %d'%(self.t_name, num)                  break              num+=1              print '\nThread(%s) released, Number: %d'%(self.t_name, num)              mylock.release()                def test():      thread1 = myThread('A')      thread2 = myThread('B')      thread1.start()      thread2.start()     if __name__== '__main__':      test()  

The code for modifying shared data is "critical section ". All "Critical Zones" must be closed between the acquire and release of the same lock object.

2. Conditional Synchronization

The lock can only provide the most basic synchronization. If a "critical section" is accessed only when some events occur, the conditional variable Condition must be used.

The Condition object is the packaging of the Lock Object. When creating a Condition object, its constructor needs a Lock object as the parameter. If this Lock object parameter is not available, condition will create an internal Rlock object. You can also call acquire and release operations on the Condition object, because the internal Lock object itself supports these operations. However, the value of Condition lies in the wait and Policy semantics provided by Condition.

How does conditional variables work? After a thread successfully obtains a condition variable, the wait () method that calls the condition variable will cause the thread to release the lock and enter the "blocked" state, until the other thread calls the notify () method of the same condition variable to wake up the thread in the "blocked" state. If you call the policyall () method of this condition variable, it will wake up all the waiting threads.

If the program or thread is always in the "blocked" State, a deadlock will occur. Therefore, if synchronization mechanisms such as locks and condition variables are used, check carefully to prevent deadlocks. For critical sections that may cause exceptions, use the finally clause in the Exception Handling Mechanism to release the lock. The thread waiting for a condition variable must wake up explicitly using the Y () method; otherwise, the thread will remain silent forever. Make sure that every wait () method call has a corresponding notify () call. Of course, you can also call the yyall () method just in case.

 

 

Producer and consumer issues are typical synchronization issues. Here we will briefly introduce two different implementation methods.

1. Condition Variables

import threading    import time    class Producer(threading.Thread):        def __init__(self, t_name):            threading.Thread.__init__(self, name=t_name)             def run(self):            global x            con.acquire()            if x > 0:                con.wait()            else:                for i in range(5):                    x=x+1                    print "producing..." + str(x)                con.notify()            print x            con.release()         class Consumer(threading.Thread):        def __init__(self, t_name):            threading.Thread.__init__(self, name=t_name)        def run(self):            global x            con.acquire()            if x == 0:                print 'consumer wait1'                con.wait()            else:                for i in range(5):                    x=x-1                    print "consuming..." + str(x)                con.notify()            print x            con.release()         con = threading.Condition()    x=0    print 'start consumer'    c=Consumer('consumer')    print 'start producer'    p=Producer('producer')         p.start()    c.start()    p.join()    c.join()    print x  

 

In the preceding example, in the initial state, the Consumer is in the wait state, and Consumer y is waiting for the Consumer five times after the Producer is continuously produced (1 operation is performed on x. Consumer is awakened to start consumption (minus 1 on x)

2. Synchronization queue

The Queue object in Python also supports thread synchronization. The Queue object can be used to implement a FIFO Queue formed by multiple producers and consumers.

The producer stores the data in the queue in sequence, and the consumer extracts the data from the queue in sequence.

 

# producer_consumer_queue    from Queue import Queue    import random    import threading    import time         #Producer thread    class Producer(threading.Thread):        def __init__(self, t_name, queue):            threading.Thread.__init__(self, name=t_name)            self.data=queue        def run(self):            for i in range(5):                print "%s: %s is producing %d to the queue!\n" %(time.ctime(), self.getName(), i)                self.data.put(i)                time.sleep(random.randrange(10)/5)            print "%s: %s finished!" %(time.ctime(), self.getName())         #Consumer thread    class Consumer(threading.Thread):        def __init__(self, t_name, queue):            threading.Thread.__init__(self, name=t_name)            self.data=queue        def run(self):            for i in range(5):                val = self.data.get()                print "%s: %s is consuming. %d in the queue is consumed!\n" %(time.ctime(), self.getName(), val)                time.sleep(random.randrange(10))            print "%s: %s finished!" %(time.ctime(), self.getName())         #Main thread    def main():        queue = Queue()        producer = Producer('Pro.', queue)        consumer = Consumer('Con.', queue)        producer.start()        consumer.start()        producer.join()        consumer.join()        print 'All threads terminate!'         if __name__ == '__main__':        main()  

In the above example, the Producer produces a "product" at random time and puts it into the queue. Consumer finds that there is a "product" in the queue and then consumes it. In this example, because the production speed of Producer is faster than the consumption speed of Consumer, Consumer consumes a product only after several "Products" are produced by Producer.

The Queue module implements a FIFO Queue that supports multiple producers and multiple consumers. Queue is useful when shared information needs to be securely exchanged among multiple threads. The default length of a Queue is unlimited, but the length can be set by setting the maxsize parameter of its constructor. The put Method of Queue is inserted at the end of the team. The prototype of this method is:

Put (item [, block [, timeout])

If the optional parameter block is true and the timeout value is None (the default value), the thread is blocked until a data unit is empty in the queue. If the timeout value is greater than 0, no data units are available during the timeout time, and Full exception is thrown. Otherwise, if the block parameter is false (the timeout parameter is ignored), item is immediately added to the idle data unit. If there is no idle data unit, Full exception is thrown.

The get method of the Queue is to retrieve data from the beginning of the Queue. Its parameters are the same as those of the put method. If the block parameter is true and the timeout value is None (the default value), the thread is blocked until there is data in the queue. If the timeout value is greater than 0 and no data is available within the timeout time, the Empty exception is thrown. Otherwise, if the block parameter is false (the timeout parameter is ignored), the data in the queue is immediately taken out. If no data is available at this time, Empty exception will also be thrown.