There is a question: the performance of multiple threads in python is very slow due to the competition of GIL. However, the performance of multiple threads on a single cpu is not obvious, but the performance of the same code is particularly obvious after it reaches multi-core. So the reason why the performance of multiple threads in multiple cores is significantly reduced is the frequent cpu switching? The main problem should be: [will frequent switching between multiple CPUs consume ?] [Note]: I have understood why GIL and GIL are required to execute only one thread at a time. The question I want to ask is that pytho has a question: the multi-thread performance of python is very slow due to the competition of GIL. However, the performance of multiple threads on a single cpu is not obvious, but the performance of the same code is particularly obvious after it reaches multi-core. So the reason why the performance of multiple threads in multiple cores is significantly reduced is the frequent cpu switching? The main problem should be: [will frequent switching between multiple CPUs consume ?]
[Note]: I have understood why GIL and GIL are required to execute only one thread at a time. The question is the execution details of python between multiple CPUs. Reply: Because python has a global interpreter lock, the thread does not support multiple CPUs. To support multiple CPUs, use multiple processes.
Python GIL
- CPython threads are the native threads of the operating system. In Linux, pthread is run, and Win thread is run on Windows by the operating system scheduling thread. A python interpreter contains a main thread and execution threads of multiple user programs. Even on a multi-core CPU platform, parallel execution of multiple threads is prohibited due to the existence of GIL.
- The multi-thread in the Python interpreter process is executed in Cooperative Multi-task mode. GIL is released when a thread encounters an I/O task. Computing-intensive (CPU-bound) threads release GIL when they execute about 100 ticks. Steps can be roughly considered as commands of Python virtual machines. The length of the time slice is irrelevant. You can use sys. setcheckinterval () to set the length of a step.
- On a single-core CPU, hundreds of interval checks can lead to a thread switching. On multi-core CPUs, thrashing occurs ).
- Python 3.2 starts to use the new GIL. In the new GIL implementation, a fixed timeout value is used to indicate that the current thread has abandoned the global lock. When the current thread keeps the lock and other threads request the lock, the current thread will be forcibly released 5 ms later.
- You can create independent processes to achieve parallelism. Python 2.6 introduces the multiprocessing multi-process package. Alternatively, you can use C/C ++ to write the key part as a Python extension, and use cytpes to directly call the export function of the dynamic library compiled by C language.
Recently I found a new method to bypass gil restrictions: using the qt Library (pyqt) in python, after the qthread class is created, it will call the qt library of c ++ to create a thread in the lower layer. Python producer thread, not limited by gil. I have studied it myself: multi-core and multi-thread are worse than single-core and multi-thread, because the single-core and multi-thread can obtain the GIL lock every time the GIL is released, so it can be executed seamlessly, however, after CPU 0 is released in multiple cores, other CPU threads will compete, but GIL may be immediately obtained by CPU 0, as a result, the threads wake up on several other CPUs will wait until the switching time ends and enter the waiting for scheduling status. This causes thread bumps and reduces system efficiency. I don't know if this is correct. Hope someone can correct me
