Examples of python multithreading usage

Examples of python multithreading usage

This article analyzes the usage of multiple threads in python. Share it with you for your reference. The details are as follows:

Today, when I tried to learn python multithreading, I suddenly found that I was not very clear about super usage. So I would like to summarize some of the problems I encountered first. When I try to write the following code:

Copy codeThe Code is as follows: class ():
Def _ init _ (self ):
Print ""
Class B ():
Def _ init _ (self ):
Super (B, self). _ init __()
# A. _ init _ (self)
Print "B"
B = B ()
Appears:

Super (B, self). _ init __()

TypeError: must be type, not classobj

Finally, we found that it was A new class in python, that is, A must be A new class. There are two solutions:

(1)
Copy codeThe Code is as follows: class A (object ):
Def _ init _ (self ):
Print ""
Class B ():
Def _ init _ (self ):
Super (B, self). _ init __()
# A. _ init _ (self) # this statement is legacy and has potential problems. Avoid using it.
Print "B"
B = B ()

(2)
Copy codeThe Code is as follows: __ metaclass __= type
Class ():
Def _ init _ (self ):
Print ""
Class B ():
Def _ init _ (self ):
Super (B, self). _ init __()
# A. _ init _ (self) # this statement is legacy and has potential problems. Avoid using it.
Print "B"
B = B ()
Note:: In super (B, self). _ init __()

When self is added to a statement, that is, super (B, self). _ init _ (self), the following error occurs:

Super (B, self). _ init _ (self)

TypeError: _ init _ () takes exactly 1 argument (2 given)

The above is just a little bit of my notes.
Copy codeThe Code is as follows: import threading, time
Class myThread (threading. Thread ):
Def _ init _ (self, threadname = ""):
# Threading. Thread. _ init _ (self, name = threadname)
Super (myThread, self). _ init _ (name = threadname)
Def run (self ):
Print "starting =", self. name, time. ctime ()
Time. sleep (5)
Print "end =", self. name, time. ctime (),
 
M = myThread ("m ")
N = myThread ("n ")
 
M. start ()
N. start ()
Output result:

Starting === m Mon Aug 08 21:55:41 2011

Starting = n Mon Aug 08 21:55:41 2011

If the main thread of a process is finished and the sub-thread is still running, the process will not exit until all sub-threads are finished. For example:
Copy codeThe Code is as follows: import threading, time
Class myThread (threading. Thread ):
Def _ init _ (self, threadname = ""):
# Threading. Thread. _ init _ (self, name = threadname)
Super (myThread, self). _ init _ (name = threadname)
Def run (self ):
Print "starting =", self. name, time. ctime ()
Time. sleep (5)
Print "end =", self. name, time. ctime (),
 
M = myThread ("m ")
M. start ()
Print "main end"
Print
The output result is:

Starting === m Mon Aug 08 22:01:06 2011

Main end

End === m Mon Aug 08 22:01:11 2011

That is, after the main process ends, the child process has not ended.

If we want to end the sub-process at the end of the main process, we should use the setDaemon () function.

Example:
Copy codeThe Code is as follows: import threading, time
Class myThread (threading. Thread ):
Def _ init _ (self, threadname = ""):
# Threading. Thread. _ init _ (self, name = threadname)
Super (myThread, self). _ init _ (name = threadname)
Def run (self ):
Print "starting =", self. name, time. ctime ()
Time. sleep (5)
Print "end =", self. name, time. ctime (),
 
M = myThread ("m ")
M. setDaemon (True)
M. start ()
Print "main end"
Print
Output result: starting === main end m Mon Aug 08 22:02:58 2011

It can be seen that the "end =…" that should be printed when the sub-process m ends is not printed ..."

Simple Thread Synchronization

Data is often shared among multiple execution threads. If only shared data is read, but multiple threads need to modify shared data, unexpected results may occur.

If both thread objects t1 and t2 need to increase the value num = 0 by 1, then t1 and t2 each modify the num 10 times, the final result of num is 20. However, when t1 obtains the num value (if num is 0 at this time), the System Schedules t1 to the "sleeping" state, and t2 to the "running" state at this time, at this time, the num value obtained by t2 is also 0, and then it assigns the value 1 of num + 1 to num. The system converts t2 to "sleeping", t1 to "running", because t1 has obtained the num value 0, so he also assigned the value of num + 1 to num 1. It was originally increased by 1 twice, but the result was that num only increased by 1. This situation may occur when multiple threads are executed simultaneously. To prevent such cases, we need to use the thread synchronization mechanism.

The simplest synchronization mechanism is "Lock"

Create a Lock Object using the threading. RLock class
Copy codeThe Code is as follows: mylock = threading. RLock ()

How to Use the lock to synchronize threads? The thread can use the lock's acquire () (obtain) method, so that the lock enters the "locked" state. Only one thread can obtain the lock at a time. If another thread tries to obtain the lock, it will be changed to the "blocked" state by the system until the thread that owns the lock calls the release () (release) method of the lock, in this way, the lock enters the "unlocked" state. A thread in the "blocked" State will receive a notification and have the right to obtain the lock. If multiple threads are in the "blocked" state, all threads will first release the "blocked" State, then the system selects a thread to obtain the lock, and other threads continue to silence ("blocked ").
Copy codeThe Code is as follows: import threading
Mylock = threading. RLock ()
Class mythread (threading. Thread)
...
Def run (self ...):
... # The code for modifying shared data cannot be placed here
Mylock. acquire ()
... # The code for modifying shared data can be placed here
Mylock. release ()
... # The code for modifying shared data cannot be placed here

We call the code for modifying shared data as a "critical section". All "critical sections" must be closed between the acquire () and release () method calls of the same lock object.

The lock can only provide the most basic synchronization level. Sometimes more complex thread synchronization is required. For example, a critical section is accessed only when some events occur (for example, when a value changes ). This requires the use of "conditional variables ".

Conditional variables are created using the threading. Condition class.
Copy codeThe Code is as follows: mycondition = threading. Condition ()

How does conditional variables work? After a thread successfully obtains a condition variable, the wait () method that calls the condition variable will cause the thread to release the lock and enter the "blocked" state, until the other thread calls the notify () method of the same condition variable to wake up the thread in the "blocked" state. If you call the policyall () method of this condition variable, it will wake up all the waiting threads.

If the program or thread is always in the "blocked" State, a deadlock will occur. Therefore, if synchronization mechanisms such as locks and condition variables are used, check carefully to prevent deadlocks. For critical sections that may cause exceptions, use the finally clause in the Exception Handling Mechanism to release the lock. The thread waiting for a condition variable must wake up explicitly using the Y () method; otherwise, the thread will remain silent forever. Make sure that every wait () method call has a corresponding notify () call. Of course, you can also call the yyall () method just in case.

Synchronization queue

We often use two threads in the producer/consumer relationship to process data in a shared buffer. For example, a producer thread accepts user data into a shared buffer and waits for a consumer thread to extract and process the data. However, if the buffer is too small and the producer and consumer asynchronous threads are not at the same speed, one thread may wait for the other. To minimize the waiting time of threads that share resources and work at the same speed, we can use a "queue" to provide additional buffers.

You can use the following code to create a "queue" object:
Copy codeThe Code is as follows: import Queue
Myqueue = Queue. Queue (maxsize = 10)
The Queue. Queue class is the synchronization implementation of a Queue. The queue length can be unlimited or limited. You can set the Queue length through the optional parameter maxsize of the Queue constructor. If maxsize is smaller than 1, the queue length is infinite.

Put a value in the queue:

Myqueue. put (10)

Call the put () method of the queue object to insert a project at the end of the team. Put () has two parameters. The first item is required and is the value of the inserted project. The second block is an optional parameter. The default value is 1. If the queue is empty and the block is 1, The put () method suspends the calling thread until a data unit is empty. If the block is 0, the put method will cause a Full exception.

Extract A value from the queue:

Myqueue. get ()

Call the get () method of the queue object to delete the queue header and return a project. The optional parameter is block. The default value is 1. If the queue is empty and the block is 1, get () will suspend the calling thread until a project is available. If the block is 0, the queue will cause an Empty exception.

We use an example to demonstrate how to use Queue:
Copy codeThe Code is as follows: # queue_example.py
From Queue import Queue
Import threading
Import random
Import time
 
# Producer thread
Class Producer (threading. Thread ):
Def _ init _ (self, threadname, queue ):
Threading. Thread. _ init _ (self, name = threadname)
Self. Fetch DATA = queue
Def run (self ):
For I in range (20 ):
Print self. getName (), 'adding', I, 'to queue'
Self. Fetch data. put (I)
Time. sleep (random. randrange (10)/10.0)
Print self. getName (), 'finished'
 
# Consumer thread
Class Consumer (threading. Thread ):
Def _ init _ (self, threadname, queue ):
Threading. Thread. _ init _ (self, name = threadname)
Self. Fetch DATA = queue
Def run (self ):
For I in range (20 ):
Print self. getName (), 'Got a value: ', self. Fetch data. get ()
Time. sleep (random. randrange (10)/10.0)
Print self. getName (), 'finished'
 
# Main thread
Def main ():
Queue = Queue ()
Producer = Producer ('producer ', queue)
Consumer = Consumer ('consumer ', queue)
 
Print 'starting threads ...'
Producer. start ()
Consumer. start ()
 
Producer. join ()
Consumer. join ()
 
Print 'all threads have terminated .'
 
If _ name _ = '_ main __':
Main ()
The output result of the program is:

Starting threads...

Producer adding 0 to queue

Consumer got a value: 0

Producer Finished

Producer adding 1 to queue

Producer Finished

Producer adding 2 to queue

Consumer Finished

Consumer got a value: 1

Consumer Finished

Consumer got a value: 2

Consumer Finished

Consumer got a value: Producer Finished

Producer adding 3 to queue

3

Consumer Finished

Consumer got a value: Producer Finished

Producer adding 4 to queue

4

ConsumerProducer Finished

ConsumerFinished

Got a value: Producer adding 5 to queue

5

Consumer Finished

Consumer got a value: Producer Finished

Producer adding 6 to queue

Producer Finished

Producer adding 7 to queue

6

Consumer Finished

Consumer got a value: 7

Producer Finished

Producer adding 8 to queue

Producer Finished

Producer adding 9 to queue

Consumer Finished

Consumer got a value: 8

ConsumerProducer FinishedFinished

 

ConsumerProducer got a value: addding 109

To queue

Producer Finished

Producer adding 11 to queue

Producer Finished

Producer adding 12 to queue

ConsumerProducer FinishedFinished

 

ConsumerProducer got a value: addding 1310

To queue

Producer Finished

Producer adding 14 to queue

Consumer Finished

Consumer got a value: 11

Producer Finished

Producer adding 15 to queue

Producer Finished

Producer adding 16 to queue

Producer Finished

Producer adding 17 to queue

Producer Finished

Producer adding 18 to queue

Consumer Finished

Consumer got a value: 12

Producer Finished

Producer adding 19 to queue

Producer Finished

Consumer Finished

Consumer got a value: 13

Consumer Finished

Consumer got a value: 14

Consumer Finished

Consumer got a value: 15

Consumer Finished

Consumer got a value: 16

Consumer Finished

Consumer got a value: 17

Consumer Finished

Consumer got a value: 18

Consumer Finished

Consumer got a value: 19

Consumer Finished

All threads have terminated.

I hope this article will help you with Python programming.