How to distinguish Python threads correctly

In the Python language, the Python thread can start to compete with the main thread on GIL from here. In t_bootstrap, after applying for GIL, that is, the sub-thread will obtain GIL, keep the status object of the active thread.

After PyEval_AcquireThread is completed, the sub-thread obtains GIL and is ready for all execution. Next, the subthread uses PyEval _ CallObjectWithKeywords to call the PyEval_EvalFrameEx that we are very familiar.

That is, the Python bytecode execution engine. The boot-> func passed to PyEval_CallObjectWithKeywords is a PyFunctionObject object, which is exactly the result of threadProc compilation defined in therad1.py. After PyEval_CallObjectWithKeywords is completed, the Child thread will release GIL and complete all the scanning and tail tasks of the destruction thread. Here, the Child thread will end.

From the code of t_bootstrap, it seems that the sub-thread will be executed until all calculations of the sub-thread are completed, and GIL will be released through PyThreadState_DeleteCurrent. In this way, will the main thread always be in the status of waiting for GIL? If this is the case, the Python thread obviously cannot support the multithreading mechanism.

In fact, in PyEval_EvalFrameEx, the simulated clock interrupt maintained in Python shown in Figure 15-2 continuously activates the thread scheduling mechanism, and continuously switches between the child thread and the main thread. So as to truly implement the multi-threaded mechanism, of course, this will be analyzed in detail later. Now we are interested in what the sub-thread has done in PyEval_AcquireThreade.

Here, I learned about PyEval_AcquireThread, and it seems that the mechanism for creating threads is clear. But in fact, there is a very important mechanism-thread state protection mechanism-hidden in a humble place: PyThreadState_New.

 
 

  
  
[Threadmodule. c]
  
  
 
  
  
Static PyObject * thread_PyThread_start_new_thread (PyObject * self, PyObject
  
  
 
  
  
* Fargs)
  
  
 
  
  
{
  
  
 
  
  
PyObject * func, * args ,*Keyw=NULL;
  
  
 
  
  
Struct bootstate * boot;
  
  
 
  
  
Long ident;
  
  
 
  
  
PyArg_UnpackTuple (fargs, "start_new_thread", 2, 3, & func, & args, & keyw );
  
  
 
  
  
// [1]: Create a bootstate Structure
  
  
 
  
  
Boot=PyMem_NEW(Struct bootstate, 1 );
  
  
 
  
  
Boot->Interp=PyThreadState_GET()->Interp;
  
  
 
  
  
Boot->FuncFunc= Func;
  
  
 
  
  
Boot->ArgsArgs= Args;
  
  
 
  
  
Boot->KeywKeyw= Keyw;
  
  
 
  
  
// [2]: Initialize the multi-threaded Environment
  
  
 
  
  
PyEval_InitThreads ();/* Start the interpreter's thread-awareness */
  
  
 
  
  
// [3]: Create a thread
  
  
 
  
  
Ident=PyThread_start_new_thread(T_bootstrap, (void *) boot );
  
  
 
  
  
Return PyInt_FromLong (ident );
  
  
 
  
  
[Thread. c]
  
  
 
  
  
/* Support for runtime thread stack size tuning.
  
  
 
  
  
A value of 0 means using the platform's default stack size
  
  
 
  
  
Or the size specified by the THREAD_STACK_SIZE macro .*/
  
  
 
  
  
Static size_t_ Pythread_stacksize=0;
  
  
 
  
  
[Thread_nt.h]
  
  
 
  
  
Long PyThread_start_new_thread (void (* func) (void *), void * arg)
  
  
 
  
  
{
  
  
 
  
  
Unsigned long rv;
  
  
 
  
  
Callobj obj;
  
  
 
  
  
Obj. id=-1;/* guilty until proved innocent */
  
  
 
  
  
Obj. func= Func;
  
  
 
  
  
Obj. arg= Arg;
  
  
 
  
  
Obj. done=CreateSemaphore(NULL, 0, 1, NULL );
  
  
 
  
  
Rv=_ Beginthread(Bootstrap, _ pythread_stacksize, & obj);/* use default stack size */
  
  
 
  
  
If (Rv= (Unsigned long)-1 ){
  
  
 
  
  
// Raw thread creation failed
  
  
 
  
  
Obj. id=-1;
  
  
 
  
  
}
  
  
 
  
  
Else {
  
  
 
  
  
WaitForSingleObject (obj. done, INFINITE );
  
  
 
  
  
}
  
  
 
  
  
CloseHandle (HANDLE) obj. done );
  
  
 
  
  
Return obj. id;
  
  
 
  
  
}
 
 

This mechanism is critical to understanding the creation and maintenance of Python threads. To analyze the thread state protection mechanism, we first need to review the thread state. In Python, each Python thread has a thread state object associated with it.

Information exclusive to each thread is recorded in the thread state object. In fact, we have seen this object before analyzing the Python initialization process. The thread state object corresponding to each thread stores the current PyFrameObject object of this thread, and the thread id contains such information. Sometimes, threads need to access this information.

For example, in the simplest case, in some cases, each thread needs to access the thread_id information stored in the thread state object. Obviously, thread A should obtain thread_id of thread, thread B is also true. If thread A obtains thread_id of thread B, it is A bad dish. This means that there must be a mechanism inside the Python thread, which is similar to the mechanism for managing processes in the operating system.

We know that when the operating system switches from process A to process B, the Context Environment of process A is saved and then switched. When process B switches back to process, the context of process A is restored, which ensures that process A is always running in its own context.

Here, the thread state object is equivalent to the context of the process. Python also has a mechanism to store and restore the thread state object. In Python, a global variable is maintained: PyThreadState * _ PyThread-State_Current.

The thread state object corresponding to the current active thread is saved in this variable. When the Python scheduling thread is executed, the thread state object corresponding to the activated thread is assigned to _ PyThreadState_Current, keep the status object of the active thread.

This raises the question: How does Python obtain the State object corresponding to the activated thread during process scheduling? Python internally uses a one-way linked list to manage the State objects of all Python threads. When you need to find a State object corresponding to a thread.

• Analysis of first-time Python deployment problems
• Powerful and quick full parsing of Python operating languages
• Rich Python debugger Resources
• A summary of Python interactive skills
• How to correctly understand Python source files

Traverse the linked list and search for its corresponding State object. In the subsequent description, the linked list is called the "State object linked list ". Next let's take a look at the key data structure implementing this mechanism in Python. Access to this state object linked list does not have to be protected by GIL.

For this state object linked list, the Python thread will create an independent lock to protect the State object linked list. The lock is created during Python initialization. PyThread_create_key creates a new key. Note that the key here is an integer, and the call in _ PyGILState_Init is the first call when PyThread_create_key is called for the first time ).

A keymutex is created through PyThread_allcate_lock. According to our previous analysis, this keymutex is actually a PNRMUTEX struct like GIL, and maintains an Event kernel object under Win32 in this struct. The keymutex function is used to mutex access to the linked list of State objects.

In _ PyGILState_Init, the new key is received by the global variable autoTLSkey maintained by Python. TLS is short for Thread Local Store, this autoTLSkey is used as a parameter for saving state objects of all threads in Python. . That is to say, the key value in all the key struct in the status object list will be autoTLSkey. Well, the reader said, you see that PyThread_create_key returns the incremental value of nkeys.

That is to say, every time you create a file, the results are different. How can we say that all keys are the same? In fact, in the Python source code, PyThread_create_key is called only in _ PyGILState_Init, and this _ PyGILState_Init is called only once during Python runtime environment initialization.