Python Multithreaded Learning material _python

Source: Internet
Author: User
one, the threads in Python use:
There are two ways to use threads in Python: Functions or wrapping thread objects with classes.
1. Function: Invoke the Start_new_thread () function in the thread module to produce a new thread. The following example:
Copy Code code as follows:

Import time
Import Thread
def timer (no, interval):
CNT = 0
While cnt<10:
print ' Thread: (%d) time:%s\n '% (No, time.ctime ())
Time.sleep (interval)
Cnt+=1
Thread.exit_thread ()

def test (): #Use thread.start_new_thread () to create 2 new threads
Thread.start_new_thread (Timer, (1,1))
Thread.start_new_thread (Timer, (2,2))
If __name__== ' __main__ ':
Test ()

The above example defines a thread function timer, which prints out 10 time records and exits, each time the interval is printed by the interval parameter. The first parameter of Thread.start_new_thread (function, args[, Kwargs]) is the thread function (the timer method in this example), the second parameter is the argument passed to the thread function, it must be the tuple type, Kwargs is an optional parameter.
The end of a thread can wait for the thread to end naturally or call the Thread.exit () or Thread.exit_thread () method in a thread function.
2, create threading. Thread to wrap a threading object, as in the following example:
Copy Code code as follows:

Import threading
Import time
Class Timer (threading. Thread): #The Timer class is derived from the class threading. Thread
def __init__ (self, num, interval):
Threading. Thread.__init__ (self)
Self.thread_num = num
Self.interval = Interval
Self.thread_stop = False
def run (self): #Overwrite run () method, put what your want the thread do
While not self.thread_stop:
print ' Thread Object (%d), time:%s\n '% (Self.thread_num, time.ctime ())
Time.sleep (Self.interval)
def stop (self):
Self.thread_stop = True

def test ():
Thread1 = Timer (1, 1)
Thread2 = Timer (2, 2)
Thread1.start ()
Thread2.start ()
Time.sleep (10)
Thread1.stop ()
Thread2.stop ()
Return
if __name__ = = ' __main__ ':
Test ()

Personally, I prefer the second way of creating my own thread class, overriding threading if necessary. Thread class methods, the control of threads can be customized by themselves.
Threading. Use of the thread class:
1, call threading in the __init__ of your own thread class. Thread.__init__ (self, name = ThreadName)
ThreadName is the name of the thread
2, run (), usually need to rewrite, write code implementation to do the required functions.
3,getname (), get the Thread object name
4,setname (), setting the thread object name
5,start (), starting thread
6,jion ([timeout]), wait for the other thread to finish before running.
7,setdaemon (BOOL) to set whether a child thread ends with the main thread and must be called before start (). The default is False.
8,isdaemon () to determine whether the thread ends with the main thread.
9,isalive () to check if the thread is running.
In addition, the threading module itself provides a number of methods and other classes that can help us better use and manage threads. Refer to http://www.python.org/doc/2.5.2/lib/module-threading.html.


Assuming that the two thread objects T1 and T2 are going to add 1 to the num=0, T1 and T2 are all modified 10 times for NUM, and the final result of Num should be 20. However, because it is multithreaded access, it is possible that the following conditions occur: in num=0, T1 gets num=0. The system at this time the T1 schedule to "sleeping" state, the T2 conversion to "running" status, T2 page to obtain num=0. T2 then adds 1 to the resulting value and assigns it to NUM, making the num=1. Then the system T2 dispatch as "sleeping", the T1 into "running". The thread t1 the 0 plus 1 that it had before and assigns it to Num. In this way, obviously T1 and T2 have completed 1 times plus 1 work, but the result is still num=1.

The case above describes one of the most common problems with multithreading: data sharing. When multiple threads have to modify a shared data, we need to synchronize the data access.

1, the simple synchronization

The simplest synchronization mechanism is "lock". Lock object by threading. The Rlock class is created. The thread can use the acquire () method of the lock to get the lock so that the lock enters the "locked" state. Only one thread can get a lock at a time. If another thread attempts to acquire the lock, the system becomes "blocked" until the thread that owns the lock calls the release () method of the lock to free the lock so that the lock enters the "unlocked" state. The "blocked" state thread receives a notification and has the right to obtain the lock. If more than one thread is in the "blocked" state, all threads will first remove the "blocked" state, and then the system chooses a thread to acquire the lock, and the other threads continue to silence ("blocked").
The thread module and lock object in Python are low-level, programmable tools available in Python, and are simple to use. As shown in the following example:
Copy Code code as follows:

Import Thread
Import time
Mylock = Thread.allocate_lock () #Allocate a lock
Num=0 #Shared Resource
def add_num (name):
Global num
While True:
Mylock.acquire () #Get the lock
# do something to the shared resource
print ' Thread%s locked! num=%s '% (name,str (num))
If Num >= 5:
print ' Thread%s released! num=%s '% (name,str (num))
Mylock.release ()
Thread.exit_thread ()
Num+=1
print ' Thread%s released! num=%s '% (name,str (num))
Mylock.release () #Release the lock.
def test ():
Thread.start_new_thread (Add_num, (' A ',))
Thread.start_new_thread (Add_num, (' B ',))
If __name__== ' __main__ ':
Test ()

Python also provides an advanced thread control library based on thread, the threading mentioned earlier. Python's threading module is a module built on thread module, which exposes many of the attributes in thread module in threading module. In thread module, Python provides a user-level thread synchronization Tool "Lock" object. In threading module, Python also provides a variant of the lock object: the Rlock object. A lock object is maintained inside the Rlock object, which is a Reentrant object. For a lock object, if a thread acquire operations two times in a row, the second acquire suspends the thread because there is no release after the first acquire. This causes the lock object to never release, causing the thread to deadlock. The Rlock object allows a thread to acquire it multiple times, because the number of threads acquire is maintained internally through a counter variable. And each acquire operation must have a release operation corresponding to it, after all release operations are completed, other threads can request the Rlock object.

Let's see how to implement synchronization using the threading Rlock object.

Copy Code code as follows:

Import threading
Mylock = Threading. Rlock ()
Num=0
Class Mythread (threading. Thread):
def __init__ (self, name):
Threading. Thread.__init__ (self)
Self.t_name = Name
def run (self):
Global num
While True:
Mylock.acquire ()
print ' \nthread (%s) locked, Number:%d '% (self.t_name, num)
If num>=4:
Mylock.release ()
print ' \nthread (%s) released, Number:%d '% (self.t_name, num)
Break
Num+=1
print ' \nthread (%s) released, Number:%d '% (self.t_name, num)
Mylock.release ()
def test ():
Thread1 = Mythread (' A ')
Thread2 = Mythread (' B ')
Thread1.start ()
Thread2.start ()
If __name__== ' __main__ ':
Test ()

We make the code that modifies the shared data a "critical section". All critical sections must be enclosed between the acquire and release of the same lock object.

2, condition synchronization

Locks provide only the most basic synchronization. If you only access a "critical section" when certain events occur, you need to use the conditional variable condition.
The condition object is a wrapper over the lock object, and when the condition object is created, its constructor requires a lock object as an argument, and if there is no parameter to the lock object, condition creates a Rlock object internally. On the Condition object, you can of course invoke acquire and release operations, because the internal lock object itself supports these operations. But the value of condition lies in the semantics of the wait and notify that it provides.
How does a conditional variable work? First after a thread succeeds in obtaining a condition variable, the Wait () method that calls this condition variable causes the thread to release the lock and go into the "blocked" state until another thread invokes the Notify () method of the same condition variable to wake the thread that enters the "blocked" state. If you call the Notifyall () method of this condition variable, all the waiting threads will be awakened.
Deadlocks can occur if a program or thread is permanently in the "blocked" state. Therefore, if the use of the lock, condition variables, such as synchronization mechanism, we must pay attention to carefully check to prevent the occurrence of deadlock. For a critical section that might produce an exception, use the finally clause in the exception handling mechanism to guarantee the release of the lock. A thread waiting for a condition variable must be awakened explicitly with the Notify () method, otherwise it will be silent forever. Make sure each wait () method call has a corresponding notify () call, and of course you can call the Notifyall () method just in case.


The problem of producer and consumer is a typical synchronization problem. Here is a brief introduction to two different implementation methods.

1, the condition variable
Copy Code code as follows:

Import threading
Import time
Class Producer (threading. Thread):
def __init__ (self, t_name):
Threading. Thread.__init__ (self, name=t_name)

def run (self):
Global X
Con.acquire ()
If x > 0:
Con.wait ()
Else
For I in range (5):
X=x+1
Print "Producing ..." + str (x)
Con.notify ()
Print X
Con.release ()

Class Consumer (threading. Thread):
def __init__ (self, t_name):
Threading. Thread.__init__ (self, name=t_name)
def run (self):
Global X
Con.acquire ()
If x = 0:
print ' Consumer wait1 '
Con.wait ()
Else
For I in range (5):
X=x-1
Print "Consuming ..." + str (x)
Con.notify ()
Print X
Con.release ()

Con = threading. Condition ()
X=0
print ' Start consumer '
C=consumer (' Consumer ')
print ' Start producer '
P=producer (' Producer ')

P.start ()
C.start ()
P.join ()
C.join ()
Print X

In the above example, in the initial state, consumer is in a wait state, producer continuous production (1 operations on X) 5 times after notify is waiting for consumer. Consumer is awakened to start consumption (minus 1 operation on X)

2, Sync queue

The queue object in Python also provides support for thread synchronization. Use the queue object to achieve a FIFO queue that is formed by multiple producers and multiple consumers.
The producer sequentially stores the data in the queue, and the consumer pulls the data from the queue in turn.
Copy Code code as follows:

# Producer_consumer_queue
From queue import queue
Import Random
Import threading
Import time
#Producer thread
Class Producer (threading. Thread):
def __init__ (self, T_name, queue):
Threading. Thread.__init__ (self, name=t_name)
Self.data=queue
def run (self):
For I in range (5):
Print '%s:%s is producing%d to the queue!\n '% (Time.ctime (), Self.getname (), i)
Self.data.put (i)
Time.sleep (Random.randrange (10)/5)
Print '%s:%s finished! '% (Time.ctime (), Self.getname ())
#Consumer thread
Class Consumer (threading. Thread):
def __init__ (self, T_name, queue):
Threading. Thread.__init__ (self, name=t_name)
Self.data=queue
def run (self):
For I in range (5):
val = Self.data.get ()
Print '%s:%s is consuming. %d in the ' queue is consumed!\n '% (Time.ctime (), Self.getname (), Val)
Time.sleep (Random.randrange (10))
Print '%s:%s finished! '% (Time.ctime (), Self.getname ())
#Main thread
def main ():
Queue = Queue ()
Producer = producer (' Pro. ', queue)
consumer = consumer (' Con. ', queue)
Producer.start ()
Consumer.start ()
Producer.join ()
Consumer.join ()
print ' All Threads terminate! '
if __name__ = = ' __main__ ':
Main ()


In the above example, producer produces a "product" in a random time and puts it into the queue. Consumer found a "product" in the queue to consume it. In this case, because producer production faster than the speed of consumer consumption, so often producer production of several "products" after consumer to consume a product.

The queue module implements a FIFO queue that supports multiple producer and multiple consumer. Queue is useful when sharing information requires a secure exchange between multiple threads. The default length of the queue is infinite, but you can set the MAXSIZE parameter of its constructor to set its length. The put method of the queue is inserted at the end of the team, and the prototype of the method is:

Put (item[, block[, timeout])

If the optional argument block is true and timeout to none (the default), the thread is block until the queue is vacated by a data unit. If the timeout is greater than 0, the full exception is thrown when there is still no available data unit for the timeout time. Conversely, if the block argument is false (ignoring the timeout parameter), item is immediately added to the free data unit, and if there is no free data unit, the full exception is thrown.

The queue's Get method is the first data from the team, and its parameters are the same as the Put method. If the block parameter is true and timeout to none (the default), the thread is blocked until there is data in the queue. If timeout is greater than 0, timeout time, there is still no desirable data, Empty exception is thrown. Conversely, if the block parameter is false (ignoring the timeout parameter), the data in the queue is immediately removed. If there is no data at this time, Empty exception will also be thrown.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.