The multithreading in Python is a multi-threading in the sense of a single CPU, and it is fundamentally different from multithreading on multiple CPUs.
Single-CPU Multithreading: Concurrency
Multi-CPU Multithreading: Concurrent internal inclusion concurrency
First, the background is emphasized:
1. What is Gil?
The full name of the Gil is the global interpreter lock, the source of which is the first consideration of Python design and the decision to make for data security.
2. Each CPU can only execute one thread at a time (multithreading in a single-core CPU is only concurrency, not parallelism, concurrency and parallelism are all the concepts of simultaneous processing of multi-channel requests at the macro level. But concurrency differs from parallelism in that two or more events occur at the same time, while concurrency refers to two or more events that occur within the same interval. )
In Python multithreading, the way each thread executes:
1. Get Gil
2. Execute the code until sleep or a Python virtual machine suspends it.
3. Release Gil
It can be seen that a thread wants to execute, must first get the Gil, we could think of Gil as a "pass", and in a Python process, the Gil has only one. A thread that does not have a pass will not be allowed to enter CPU execution.
...
Py_begin_allow_threads
Sleep ((int) secs);
Py_end_allow-threads
...
This code is the sleep code that releases the Gil through a macro before performing sleep, and then acquires the Gil before the rest of the code is executed. Other operations, such as IO, can also have similar operations, so that the IO-intensive programs, or the use of C libraries to calculate the program, or to a large extent to avoid the Gil to achieve the effect of thread parallelism. But for the Pure Python Code program, Gil is afraid to hide.
Another question is how the Gil is released, and we see that macros are available for release in the PYTHON/C API, so what about normal python statements? The interpreter forces the Gil to be freed after executing the 100 Python code, which allows other threads to execute
The last thing to note is that the Gil problem is the interpreter-related, not the language-related. This means that it is only an implementation of the Python language interpreter, not the language itself. In fact, the Gil is a very coarse-grained lock of the interpreter, we can use a finer-grained lock to increase parallelism, and Gindo wrote a patch to cancel the Gil, but as it turns out, a fine-grained lock results in a twice-fold reduction in the performance of a single-threaded application, So finally decided to prioritize the performance of the single-threaded, continue to retain the Gil. But the other two branches of Python, Jython and IronPython, do not have the Gil problem, which enables thread parallelism. Why is it? There's no reason for that.
Python pursues simplicity and simplicity. Adding multithreading to your code is often a simple matter. This violates the cicada inside Python. Providing multithreading does not mean that he is a regular understanding of multithreading. Like Ruby, Ruby also provides multithreading but is really pseudo-threading.
1. To be precise, only the thread that obtained the Gil can execute the Python bytecode, that is, the Python bytecode cannot run in parallel on multiple cores. But Python has a lot of C code that can be protected from the Gil.
2. Who says parallelism must be multi-threaded? UNIX and even windows were doing well in parallel when there were no threads, with TSR under DOS. Even pure python, you can use multiprocess to do parallel, the same can run the CPU full.
Why the multithreading of Python cannot take advantage of multicore CPUs, but when we write code, multithreading is indeed concurrency, and faster than a single thread.