This article is based on py2.x
Thread
Multitasking can be done by multiple processes or by multithreading within a process.
We mentioned earlier that a process is made up of several threads, and a process has at least one thread.
Multithreading is similar to executing several different programs at the same time, multithreaded operation has the following advantages:
- You can put long-running tasks in the background to deal with.
- User interface can be more attractive, such as the user clicked a button to trigger the processing of certain events, you can pop up a progress bar to show the progress of processing.
- The program may run faster.
- In some of the tasks that need to wait for implementation, such as user input, file read and write, and network send and receive data, thread is more useful. In this case we can release some precious resources, such as memory footprint.
Python's standard library provides two modules: the thread 和threading low- thread level module, the threading Advanced module, and the thread encapsulation. In most cases, we only need to use the Advanced module threading.
Starting a thread is to pass a function in and create a thread instance, and then call Start () to start execution:
#-*-Coding:utf-8-*-import time, threading# code executed by the new thread: Def loop (): print ' thread%s is running ... '% Threadi Ng.current_thread (). Name n = 0 while n < 5:n = n + 1 print ' thread%s >>>%s '% (Threadin G.current_thread (). Name, N) time.sleep (1) print ' thread%s ended. '% Threading.current_thread (). Nameprint ' Threa D%s is running ... '% threading.current_thread (). Namet = Threading. Thread (Target=loop, name= ' Loopthread ') T.start () t.join () print ' thread%s ended. '% Threading.current_thread (). Name gets: Thread Mainthread is Running...thread loopthread is running...thread loopthread >>> 1thread loopthread &G t;>> 2thread loopthread >>> 3thread loopthread >>> 4thread loopthread >>> 5thread loopth Read Ended.thread Mainthread ended.
Since any process will start a thread by default, we call it the main thread, and the main thread can start new threads, and the Python threading module has a current_thread () function that always returns an instance of the current thread. The primary thread instance is named Mainthread, and the child thread name is specified at creation time, and we use Loopthread to name the child thread. The name is only used to display when printing, there is absolutely no other meaning, if you do not name Python will automatically give the thread named Thread-1,thread-2 ...
Lock
The biggest difference between multithreading and multi-process is that, in many processes, the same variable, each one copy is in each process, does not affect each other, and in many threads, all variables are shared by all threads, so any one variable can be modified by any one thread, so, The biggest danger of sharing data between threads is that multiple threads change a variable at the same time, and the content is scrambled.
import time, threading# 假定这是你的银行存款:balance = 0def change_it(n): # 先存后取,结果应该为0: global balance balance = balance + n balance = balance - ndef run_thread(n): for i in range(100000): change_it(n)t1 = threading.Thread(target=run_thread, args=(5,))t2 = threading.Thread(target=run_thread, args=(8,))t1.start()t2.start()t1.join()t2.join()print balance得到:46,且每次运行结果都会不一样。
We define a shared variable balance, the initial value is 0, and start two threads, the first to save after fetching, the theoretical result should be 0, but, because the thread scheduling is determined by the operating system, when T1, T2 alternating execution, as long as the number of cycles, balance the result is not necessarily 0.
Because of the alternation between each other, so the result will change, if it is in the bank operation, a deposit can cause the balance is not correct, so you must ensure that a thread when modifying balance, other threads must not be changed.
If we want to make sure that the balance calculation is correct, we need to give change_it () a lock, and when a thread starts executing change_it (), we say that the thread has acquired a lock, so other threads cannot execute change_it () at the same time, wait until the lock is released , the lock can be changed after it is acquired. Since there is only one lock, no matter how many threads, at most one thread at a time holds the lock, so there is no conflict of modification. Creating a lock is through threading. Lock () to achieve:
The Modified code:
# -*- coding:utf-8 -*-import time, threading# 假定这是你的银行存款:balance = 0def change_it(n): # 先存后取,结果应该为0: global balance balance = balance + n balance = balance - nlock = threading.Lock()def run_thread(n): for i in range(100000): # 先要获取锁: lock.acquire() try: # 放心地改吧: change_it(n) finally: # 改完了一定要释放锁: lock.release()t1 = threading.Thread(target=run_thread, args=(5,))t2 = threading.Thread(target=run_thread, args=(8,))t1.start()t2.start()t1.join()t2.join()print balance结果,无论怎么执行都是0,这正是我们期望的结果。
When multiple threads execute Lock.acquire () at the same time, only one thread can successfully acquire the lock and then continue executing the code, and the other threads continue to wait until the lock is acquired.
The thread that gets the lock must release the lock after it is exhausted, or the thread that waits for the lock waits forever to become a dead thread. So we use try...finally to make sure that the lock will be released.
The advantage of the lock is to ensure that a certain section of the key code can only be performed by a single thread from start to finish, the disadvantage is also many, the first is to prevent multi-threaded concurrent execution, a piece of code containing the lock can actually only be executed in single-threaded mode, the efficiency is greatly reduced. Second, because multiple locks can exist, different threads hold different locks, and attempting to acquire a lock held by the other side can cause deadlocks, cause multiple threads to hang all, neither execute nor end, only by the operating system to force termination.
Global Interpreter
If you unfortunately have a multicore CPU, you must be thinking that multicore should be able to execute multiple threads at the same time.
In Python's original interpreter CPython there is the Gil (Global interpreter lock, Universal interpreter lock) So when interpreting the execution of the Python code, a mutex is created that restricts the thread's access to the shared resource until the interpreter encounters the i/ O Member release Gil when operation or operation times reach certain data. Because of the existence of the global lock, in the multi-threaded operation, can not call multiple CPU cores, only one kernel, so in the CPU-intensive operation, the use of multi-threading, more inclined to multi-process, then multi-threading suitable for what kind of application scenario? For IO-intensive operations, multithreading can significantly improve efficiency, such as the development of Python crawler, most of the time crawler is waiting for the socket to return data, network IO operation latency is much larger than the CPU.
ThreadLocal
In a multithreaded environment, each thread has its own data. It is better for a thread to use its own local variables than to use global variables, because local variables can only be seen by the thread themselves and not affect other threads, and changes to global variables must be locked.
But local variables also have a problem, that is, when the function is called, it is cumbersome to pass:
def process_student(name): std = Student(name) # std是局部变量,但是每个函数都要用它,因此必须传进去: do_task_1(std) do_task_2(std)def do_task_1(std): do_subtask_1(std) do_subtask_2(std)def do_task_2(std): do_subtask_2(std) do_subtask_2(std)
Every single layer of the function calls all the arguments. With global variables? No, because each thread handles different student objects and cannot be shared.
What if a global dict is used to store all the student objects, and then the thread itself as the key to get the student object corresponding to the threads?
global_dict = {}def std_thread(name): std = Student(name) # 把std放到全局变量global_dict中: global_dict[threading.current_thread()] = std do_task_1() do_task_2()def do_task_1(): # 不传入std,而是根据当前线程查找: std = global_dict[threading.current_thread()] ...def do_task_2(): # 任何函数都可以查找出当前线程的std变量: std = global_dict[threading.current_thread()] ...
This approach is theoretically feasible, and its greatest advantage is that it eliminates the problem of STD objects passing through each layer of functions, but the code that gets STD for each function is a bit ugly.
Is there a simpler way?
Threadlocal came into being, do not find dict,threadlocal help you do this automatically:
import threading# 创建全局ThreadLocal对象:local_school = threading.local()def process_student(): print ‘Hello, %s (in %s)‘ % (local_school.student, threading.current_thread().name)def process_thread(name): # 绑定ThreadLocal的student: local_school.student = name process_student()t1 = threading.Thread(target= process_thread, args=(‘Alice‘,), name=‘Thread-A‘)t2 = threading.Thread(target= process_thread, args=(‘Bob‘,), name=‘Thread-B‘)t1.start()t2.start()t1.join()t2.join()得到:Hello, Alice (in Thread-A)Hello, Bob (in Thread-B)
A global variable local_school is an ThreadLocal object, each Thread of which can read student and write properties, but does not affect each other. You can think of it local_school as a global variable, but each property, such as local_school.student a thread's local variable, can read and write without interfering with each other, and does not have to manage locks, threadlocal internal processing.
Can be understood as a global variable local_school is a dict, not only can be used local_school.student , you can also bind other variables, such as local_school.teacher and so on.
ThreadLocalThe most common place is to bind a database connection for each thread, HTTP requests, user identity information, and so on, so that all the calls to the processing functions of a thread can be easily accessed by these resources.
The Python thread