37 Multi-Threading

Multitasking can be done by multiple processes or by multithreading within a process.

We mentioned earlier that a process is made up of several threads, and a process has at least one thread.

Because threads are execution units that are directly supported by the operating system, high-level languages often have built-in multithreading support, Python is no exception, and Python threads are real POSIX threads, not those that are emulated.

Python's standard library provides two modules: _thread and threading , is the _thread low-level module, threading is the Advanced module, the _thread encapsulation. In most cases, we only need to use threading this advanced module.

Starting a thread is to pass a function in and create an Thread instance, and then call to start() start executing:

import time, threading# code executed by the new thread: Def loop (): Print ('thread%s is running ...'%Threading.current_thread (). Name) n=0     whileN <5: N= n +1Print ('Thread%s >>>%s'%(Threading.current_thread (). Name, N)) Time.sleep (1) Print ('thread%s ended.'%Threading.current_thread (). Name) print ('thread%s is running ...'%Threading.current_thread (). Name) T= Threading. Thread (Target=loop, name='Loopthread') T.start () T.join () print ('thread%s ended.'% Threading.current_thread (). Name)

The results of the implementation are as follows:

 is is 12 3 4   5thread Loopthread ended.thread mainthread ended.

Since any process will start a thread by default, we call it the main thread, the main thread can start new threads, and the Python threading module has a current_thread() function that always returns an instance of the current thread. The name of the main thread instance is called, the name of the MainThread child thread is specified at creation time, and we use the LoopThread named child thread. The name is only used to display when printing, there is absolutely no other meaning, if not the name of Python automatically named to Thread-1 the thread, Thread-2 ...

Lock

The biggest difference between multithreading and multi-process is that, in many processes, the same variable, each one copy is in each process, does not affect each other, and in many threads, all variables are shared by all threads, so any one variable can be modified by any one thread, so, The biggest danger of sharing data between threads is that multiple threads change a variable at the same time, and the content is scrambled.

Take a look at how multiple threads can manipulate a variable to change the content:

 import time, threading# assume this is your bank deposit: Balance  = 0  def change_it (N): # First Save and then fetch, the result should be 0:  global   balance balance  = balance + n balance  = b Alance- Ndef Run_thread (n):  for  i in  Range (100000  ): Change_it (n) T1 
    = Threading. Thread (Target=run_thread, args= (5   = Threading. Thread (Target=run_thread, args= (8   

We define a shared variable balance , the initial value is 0 , and start two threads, save after fetch, theoretically the result should be 0 , however, because the thread scheduling is determined by the operating system, when T1, T2 alternating execution, as long as the number of loops, balance the result is not necessarily 0up.

The reason is because a statement in a high-level language is a few statements when the CPU executes, even if a simple calculation:

Balance = balance + N

Also in two steps:

1. To be calculated balance + n and deposited into a temporary variable;
2. Assigns the value of the temporary variable balance .

which can be seen as:

x = balance += x

Since x is a local variable, two threads each have their own x, when the code executes normally:

Initial value balance =0t1:x1= Balance +5# x1 =0+5=5t1:balance= x1 # balance =5t1:x1= Balance-5# x1 =5-5=0t1:balance= x1 # balance =0t2:x2= Balance +8# x2 =0+8=8t2:balance= x2 # balance =8t2:x2= Balance-8# x2 =8-8=0t2:balance= x2 # balance =0Result Balance=0

However, T1 and T2 are run alternately if the operating system performs T1, T2 in the following order:

Initial value balance =0t1:x1= Balance +5# x1 =0+5=5t2:x2= Balance +8# x2 =0+8=8t2:balance= x2 # balance =8t1:balance= x1 # balance =5t1:x1= Balance-5# x1 =5-5=0t1:balance= x1 # balance =0t2:x2= Balance-8# x2 =0-8= -8t2:balance= x2 # balance =-8Result Balance= -8

The reason is that because the modification balance requires multiple statements, while executing these statements, the thread may break, causing multiple threads to change the contents of the same object.

When two threads are saved at the same time, it may cause the balance to be incorrect, and you certainly don't want your bank account to become a negative number somehow, so we have to make sure that when a thread is modified balance , other threads must not change.

If we want to make sure that the balance calculation is correct, we will give change_it() a lock, and when a thread starts executing change_it() , we say that the thread has acquired a lock, so the other thread cannot execute at the same time, change_it() wait until the lock is released, and then the lock can be changed. Since there is only one lock, no matter how many threads, at most one thread at a time holds the lock, so there is no conflict of modification. Creating a lock is accomplished by threading.Lock() :

0 Lock = Threading. Lock () def run_thread (n):    for in range (100000):        # to get the lock first        :   Lock. Acquire ()        try:            # change it with confidence:            change_it (n)          Finally:            # change it. Be sure to release            the Lock:lock. Release ()

When multiple threads are executing at the same time lock.acquire() , only one thread can successfully acquire the lock and then continue executing the code, and the other threads continue to wait until the lock is acquired.

The thread that gets the lock must release the lock after it is exhausted, or the thread that waits for the lock waits forever to become a dead thread. So we use it try...finally to make sure that the lock will be released.

The advantage of the lock is to ensure that a certain section of the key code can only be performed by a single thread from start to finish, the disadvantage is also many, the first is to prevent multi-threaded concurrent execution, a piece of code containing the lock can actually only be executed in single-threaded mode, the efficiency is greatly reduced. Second, because multiple locks can exist, different threads hold different locks, and attempting to acquire a lock held by the other side can cause deadlocks, cause multiple threads to hang all, neither execute nor end, only by the operating system to force termination.

Multi-core CPUs

If you unfortunately have a multicore CPU, you must be thinking that multicore should be able to execute multiple threads at the same time.

What would happen if we wrote a dead loop?

Turn on Activity Monitor for Mac OS X, or Windows task Manager, to monitor the CPU usage of a process.

We can monitor a dead loop thread that consumes a CPU 100%.

If there are two dead loop threads, in the multi-core CPU, you can monitor the CPU that consumes 200%, that is, two CPU cores.

To fully run the core of the N-core CPU, you must start n dead loop threads.

Try Python to write a dead loop:

import threading, Multiprocessingdef loop ():     0     while True:         1  for inch Range (Multiprocessing.cpu_count ()):     = Threading. Thread (target=Loop)    T.start ()

Starting with the same number of n threads as the CPU cores, you can monitor the CPU occupancy by only 102% on a 4-core CPU, that is, only one core is used.

But using C, C + + or Java to rewrite the same dead loop, can directly complete the core, 4 cores ran to 400%, 8 cores ran to 800%, why not python?

Because the python thread is a real thread, but the interpreter executes the code, there is a Gil Lock: Global interpreter lock, before any Python thread executes, must first obtain the Gil Lock, and then, each execution of 100 bytecode, the interpreter will automatically release the Gil lock , giving other threads a chance to execute. This Gil global lock actually locks the execution code of all threads, so multithreading can only be performed alternately in Python, even if 100 threads run on a 100-core CPU, only 1 cores are used.

Gil is a legacy of the Python interpreter design, usually the interpreter we use is an officially implemented CPython, to really take advantage of multicore unless you rewrite an interpreter with no Gil.

So, in Python, you can use multi-threading, but don't expect to make efficient use of multicore. If you must use multi-core through multithreading, that can only be achieved through C extension, but this will lose the Python simple and easy to use features.

However, there is no need to worry too much that Python cannot use multithreading to achieve multicore tasks, but it can achieve multi-core tasks through multiple processes. Multiple Python processes have separate Gil locks that do not affect each other.

Summary

Multithreaded programming, model complex, prone to conflict, must be isolated with locks, but also beware of the occurrence of deadlocks.

The Python interpreter was designed with a Gil global lock, resulting in multi-threaded inability to take advantage of multicore. Multi-threaded concurrency is a beautiful dream in Python.

multi_threading.py

do_lock.py

37 Multi-Threading