Analysis of Python multithreading and python

Analysis of Python multithreading and python

I read several blogs today, mainly explaining thread instances and how to avoid competition between threads. I feel that it is useful to myself. So I will write it first for future reference.

Example 1: We will request three different url1. single threads:

1 import time 2 from urllib. request import urlopen 3 4 5 def get_responses (): 6 urls = [7 'HTTP: // www.baidu.com ', 8 'HTTP: // www.taobao.com', 9 'HTTP: // www.alibaba.com ', 10] 11 start = time. time () 12 for url in urls: 13 print (url) 14 resp = urlopen (url) 15 print (resp. getcode () # obtain the status code 16 print ("spent time: % s" % (time. time ()-start) 17 18 get_responses ()

Explanation:
Request in url order
The cpu does not request the next url unless it returns a response from a url.
Network requests take a long time, so the cpu remains idle for the time the network request returns.

Output:
Http://www.baidu.com
200
Http://www.taobao.com
200
Http://www.alibaba.com
200
Spent time: 1.1927924156188965

2. multithreading:

from urllib.request import urlopenimport timefrom threading import Threadclass GetUrlThread(Thread):    def __init__(self, url):        self.url = url        super(GetUrlThread, self).__init__()    def run(self):        resp = urlopen(self.url)        print(self.url, resp.getcode())def get_responses():    urls = [        'http://www.baidu.com',        'http://www.taobao.com',        'http://www.alibaba.com',    ]    start = time.time()    threads = []    for url in urls:        t = GetUrlThread(url)        threads.append(t)        t.start()    for t in threads:        t.join()    print("spent time:%s" % (time.time()-start))get_responses()

Explanation:
Realized the improvement of program execution time
We have written a multi-threaded program to reduce the cpu wait time. When we are waiting for a network request in a thread to return, the cpu can switch to another thread for network requests in other threads.
We expect a thread to process a url, So we pass a url when instantiating the Thread class.
Thread running means the run () method in the execution class.
In any case, we want each thread to execute run ().
Create a thread for each url and call the start () method, which indicates that the cpu can execute the run () method in the thread.
The join () method is called because we want all threads to calculate the time consumed when execution is complete.
Join () can notify the main thread to wait until the end of this thread before the next instruction can be executed.
We call the join () method for each thread, So we calculate the running time after all threads are executed.
Thread:
The cpu may not execute the run () method immediately after calling start.
You cannot determine the execution sequence of run () between different threads.
For a separate thread, you can ensure that the statements in the run () method are executed in order.
This is because the url in the thread is requested first and then the returned result is printed.

Output:
Http://www.baidu.com 200
Http://www.alibaba.com 200
Http://www.taobao.com 200
Spent time: 0.6294200420379639

Example 2: race condition 1. BUG version:

from threading import Threadimport time#define a global variablesome_var = 0class IncrementThread(Thread):    def run(self):        # we want to read a global variable        # and then increment it        global some_var        read_var = some_var        print("some_var in %s is %d" % (self.name, read_var))        time.sleep(0.1)        some_var = read_var + 1        print("some_var in %s is %d" % (self.name, some_var))def use_increment_thread():    threads = []    for i in range(50):        t = IncrementThread()        threads.append(t)        t.start()    for t in threads:        t.join()    print("After 50 modifications, some_var should have become 50")    print("After 50 modifications, some_var is %d" % some_var)use_increment_thread()

Explanation:

There is a global variable that all threads want to modify.
All threads should add 1 to the global variable.
There are 50 threads, and the final value should be 50, but it does not.
Why didn't it reach 50?
When some_var is 15, thread t1 reads some_var. At this time, the cpu gives control to another thread t2.
Some_var read by thread t2 is also 15
Both t1 and t2 add some_var to 16
What we expected at the time was that the two t1 t2 threads changed some_var + 2 to 17.
Here there is resource competition.
The same situation may also occur between other threads, so the final result is less than 50.
Output:
Some_var in Thread-1 is 0
Some_var in Thread-2 is 0
Some_var in Thread-3 is 0
Some_var in Thread-4 is 0
Some_var in Thread-5 is 0
Some_var in Thread-6 is 0
Some_var in Thread-7 is 0
Some_var in Thread-8 is 0
Some_var in Thread-9 is 0
Some_var in Thread-10 is 0
Some_var in Thread-11 is 0
Some_var in Thread-12 is 0
Some_var in Thread-13 is 0
Some_var in Thread-14 is 0
Some_var in Thread-15 is 0
Some_var in Thread-16 is 0
Some_var in Thread-17 is 0
Some_var in Thread-18 is 0
Some_var in Thread-19 is 0
Some_var in Thread-20 is 0
Some_var in Thread-21 is 0
Some_var in Thread-22 is 0
Some_var in Thread-23 is 0
Some_var in Thread-24 is 0
Some_var in Thread-25 is 0
Some_var in Thread-26 is 0
Some_var in Thread-27 is 0
Some_var in Thread-28 is 0
Some_var in Thread-29 is 0
Some_var in Thread-30 is 0
Some_var in Thread-31 is 0
Some_var in Thread-32 is 0
Some_var in Thread-33 is 0
Some_var in Thread-34 is 0
Some_var in Thread-35 is 0
Some_var in Thread-36 is 0
Some_var in Thread-37 is 0
Some_var in Thread-38 is 0
Some_var in Thread-39 is 0
Some_var in Thread-40 is 0
Some_var in Thread-41 is 0
Some_var in Thread-42 is 0
Some_var in Thread-43 is 0
Some_var in Thread-44 is 0
Some_var in Thread-45 is 0
Some_var in Thread-46 is 0
Some_var in Thread-47 is 0
Some_var in Thread-48 is 0
Some_var in Thread-49 is 0
Some_var in Thread-50 is 0
Some_var in Thread-6 is 1
Some_var in Thread-5 is 1
Some_var in Thread-2 is 1
Some_var in Thread-4 is 1
Some_var in Thread-1 is 1
Some_var in Thread-3 is 1
Some_var in Thread-12 is 1
Some_var in Thread-13 is 1
Some_var in Thread-11 is 1
Some_var in Thread-10 is 1
Some_var in Thread-9 is 1
Some_var in Thread-7 is 1
Some_var in Thread-8 is 1
Some_var in Thread-21 is 1
Some_var in Thread-20 is 1
Some_var in Thread-19 is 1
Some_var in Thread-18 is 1
Some_var in Thread-17 is 1
Some_var in Thread-15 is 1
Some_var in Thread-14 is 1
Some_var in Thread-16 is 1
Some_var in Thread-26 is 1
Some_var in Thread-25 is 1
Some_var in Thread-24 is 1
Some_var in Thread-22 is 1
Some_var in Thread-23 is 1
Some_var in Thread-31 is 1
Some_var in Thread-29 is 1
Some_var in Thread-28 is 1
Some_var in Thread-27 is 1
Some_var in Thread-30 is 1
Some_var in Thread-38 is 1
Some_var in Thread-37 is 1
Some_var in Thread-36 is 1
Some_var in Thread-35 is 1
Some_var in Thread-32 is 1
Some_var in Thread-33 is 1
Some_var in Thread-34 is 1
Some_var in Thread-44 is 1
Some_var in Thread-43 is 1
Some_var in Thread-42 is 1
Some_var in Thread-41 is 1
Some_var in Thread-40 is 1
Some_var in Thread-39 is 1
Some_var in Thread-50 is 1
Some_var in Thread-49 is 1
Some_var in Thread-48 is 1
Some_var in Thread-47 is 1
Some_var in Thread-45 is 1
Some_var in Thread-46 is 1
After 50 modifications, some_var shoshould have become 50
After 50 modifications, some_var is 1

To solve the competition with locks:

 1 from threading import Lock, Thread 2 import time 3 lock = Lock() 4 some_var = 0 5  6 class IncrementThread(Thread): 7     def run(self): 8         #we want to read a global variable 9         #and then increment it10         global some_var11         lock.acquire()12         read_value = some_var13         print("some_var in %s is %d" % (self.name, read_value))14         time.sleep(0.1)15         some_var = read_value + 116         print("some_var in %s after increment is %d" % (self.name, some_var))17         lock.release()18 19 def use_increment_thread():20     threads = []21     for i in range(50):22         t = IncrementThread()23         threads.append(t)24         t.start()25     for t in threads:26         t.join()27     print("After 50 modifications, some_var should have become 50")28     print("After 50 modifications, some_var is %d" % (some_var,))29 30 use_increment_thread()

Explanation:

Lock is used to prevent competition conditions.
If thread t1 obtains the lock before performing some operations. Other threads do not perform the same operation before t1 releases Lock.
What we want to determine is that once thread t1 has read some_var, it will not be able to read some_var until T1.
In this way, some_var is read and modified as an atomic operation in logic.
Output:
Some_var in Thread-1 is 0
Some_var in Thread-1 after increment is 1
Some_var in Thread-2 is 1
Some_var in Thread-2 after increment is 2
Some_var in Thread-3 is 2
Some_var in Thread-3 after increment is 3
Some_var in Thread-4 is 3
Some_var in Thread-4 after increment is 4
Some_var in Thread-5 is 4
Some_var in Thread-5 after increment is 5
Some_var in Thread-6 is 5
Some_var in Thread-6 after increment is 6
Some_var in Thread-7 is 6
Some_var in Thread-7 after increment is 7
Some_var in Thread-8 is 7
Some_var in Thread-8 after increment is 8
Some_var in Thread-9 is 8
Some_var in Thread-9 after increment is 9
Some_var in Thread-10 is 9
Some_var in Thread-10 after increment is 10
Some_var in Thread-11 is 10
Some_var in Thread-11 after increment is 11
Some_var in Thread-12 is 11
Some_var in Thread-12 after increment is 12
Some_var in Thread-13 is 12
Some_var in Thread-13 after increment is 13
Some_var in Thread-14 is 13
Some_var in Thread-14 after increment is 14
Some_var in Thread-15 is 14
Some_var in Thread-15 after increment is 15
Some_var in Thread-16 is 15
Some_var in Thread-16 after increment is 16
Some_var in Thread-17 is 16
Some_var in Thread-17 after increment is 17
Some_var in Thread-18 is 17
Some_var in Thread-18 after increment is 18
Some_var in Thread-19 is 18
Some_var in Thread-19 after increment is 19
Some_var in Thread-20 is 19
Some_var in Thread-20 after increment is 20
Some_var in Thread-21 is 20
Some_var in Thread-21 after increment is 21
Some_var in Thread-22 is 21
Some_var in Thread-22 after increment is 22
Some_var in Thread-23 is 22
Some_var in Thread-23 after increment is 23
Some_var in Thread-24 is 23
Some_var in Thread-24 after increment is 24
Some_var in Thread-25 is 24
Some_var in Thread-25 after increment is 25
Some_var in Thread-26 is 25
Some_var in Thread-26 after increment is 26
Some_var in Thread-27 is 26
Some_var in Thread-27 after increment is 27
Some_var in Thread-28 is 27
Some_var in Thread-28 after increment is 28
Some_var in Thread-29 is 28
Some_var in Thread-29 after increment is 29
Some_var in Thread-30 is 29
Some_var in Thread-30 after increment is 30
Some_var in Thread-31 is 30
Some_var in Thread-31 after increment is 31
Some_var in Thread-32 is 31
Some_var in Thread-32 after increment is 32
Some_var in Thread-33 is 32
Some_var in Thread-33 after increment is 33
Some_var in Thread-34 is 33
Some_var in Thread-34 after increment is 34
Some_var in Thread-35 is 34
Some_var in Thread-35 after increment is 35
Some_var in Thread-36 is 35
Some_var in Thread-36 after increment is 36
Some_var in Thread-37 is 36
Some_var in Thread-37 after increment is 37
Some_var in Thread-38 is 37
Some_var in Thread-38 after increment is 38
Some_var in Thread-39 is 38
Some_var in Thread-39 after increment is 39
Some_var in Thread-40 is 39
Some_var in Thread-40 after increment is 40
Some_var in Thread-41 is 40
Some_var in Thread-41 after increment is 41
Some_var in Thread-42 is 41
Some_var in Thread-42 after increment is 42
Some_var in Thread-43 is 42
Some_var in Thread-43 after increment is 43
Some_var in Thread-44 is 43
Some_var in Thread-44 after increment is 44
Some_var in Thread-45 is 44
Some_var in Thread-45 after increment is 45
Some_var in Thread-46 is 45
Some_var in Thread-46 after increment is 46
Some_var in Thread-47 is 46
Some_var in Thread-47 after increment is 47
Some_var in Thread-48 is 47
Some_var in Thread-48 after increment is 48
Some_var in Thread-49 is 48
Some_var in Thread-49 after increment is 49
Some_var in Thread-50 is 49
Some_var in Thread-50 after increment is 50
After 50 modifications, some_var shoshould have become 50
After 50 modifications, some_var is 50

Example 3: atomic operation BUG version in a multi-threaded environment:

 1 from threading import Thread 2 import time 3  4 class CreateListThread(Thread): 5     def run(self): 6         self.entries = [] 7         for i in range(10): 8             # time.sleep(0.1) 9             self.entries.append(i)10         for each in self.entries:11             print(each, end = " ")12             time.sleep(0.1)13 14 def use_create_list_thread():15     for i in range(3):16         t = CreateListThread()17         t.start()18 19 use_create_list_thread()

 

 

Explanation:
When a thread is being printed, the cpu switches to another thread, which leads to incorrect results. We need to ensure that print self. entries is a logical atomic operation to prevent printing from being interrupted by other threads.
Because the printing speed is too fast, I intentionally enlarged this time and added a time. sleep (0.1)
Output:
0 0 0 1 1 1 2 2 2 3 3 4 4 5 5 5 6 6 6 7 7 8 8 8 9 9

2. Lock to ensure the atomicity of the operation

 1 from threading import Thread, Lock 2 import time 3  4 lock = Lock() 5  6  7 class CreateListThread(Thread): 8     def run(self): 9         self.entries = []10         for i in range(10):11             time.sleep(0.1)12             self.entries.append(i)13         lock.acquire()14         for each in self.entries:15             print(each, end = " ")16             time.sleep(0.1)17         lock.release()18 19 20 def use_create_list_thread():21     for i in range(3):22         t = CreateListThread()23         t.start()24 25 use_create_list_thread()

Output:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 4 5 6 7 8 9