Analysis on Python virtual machine execution framework

A core concept in the Python virtual machine. In the Python language, everything is an object, that is, an integer is an object, and a string is also an object. In fact, the type is also an object, an integer is an object.

Understanding these two points is sufficient for the second part. However, virtual machines and execution environments are only part of the Python operating mechanism or running model. In order to fully understand the entire operating mechanism of Python, we also need to get a general idea about the runtime environment of Python.

As we mentioned above, PyFrameObject corresponds to the stack frame of the executable file during execution, but an executable file must be run in the operating system, and only the stack frame is not enough. We have omitted two concepts that are critical to the running of executable files: Processes and threads.

In this section, we first need to understand the overall concept of the Python runtime model (mainly the thread model, although this part of content will be further investigated when we analyze the multi-threaded Implementation of Python. However, the Python virtual machine will create a main thread during initialization.

Therefore, there is a main thread in the runtime environment, and the exception mechanism of Python analyzed in this section will use the internal thread model of Python, therefore, it is also necessary to have an overall conceptual understanding of the Python thread model. Taking the Win32 platform as an example, we know that for Native Win32 executable files, whether produced by C/C ++ or by Delphi, they will all run in a Process.

The process is not an activity object corresponding to the machine instruction sequence. The activity object corresponding to the machine instruction sequence in the executable file is abstracted by the Thread, the process is the active environment of the thread. For a single-threaded executable file, the operating system will create a process during execution, and there will be another main thread in the process. For a multi-threaded executable file, during execution, the operating system creates one process and multiple threads.

The multiple threads can share the global variables in the process address space, which naturally leads to the thread synchronization problem. The switching between the CPU and the task is actually switching between threads. When switching the task, the CPU needs to save the thread environment, and after switching to the new thread, the thread environment that needs to be restored.

These concepts about program running are also applicable to Python. Python supports multiple threads, and a thread in Python is a native thread in the operating system. Here we do not go deep into the multithreading mechanism. Now we only need to remember that multiple threads may exist during Python execution.

We have seen the general running framework of virtual machines. In fact, this virtual machine is the abstraction of CPU in Python and can be seen as a soft CPU, all threads in Python use this soft CPU for computing. The task switching mechanism on the real machine corresponds to Python.

Is to make different threads take turns using the virtual machine mechanism. The thread runtime environment needs to be saved during CPU switching tasks. For Python, you also need to save information about the current thread before switching the thread. In Python, the abstract of thread state information is implemented through the PyThreadState object. A thread will have a PyThreadState object.

So in another sense, this PyThreadState object can also be seen as an abstraction of the thread itself. But in fact, there is a big difference between the two. PyThreadState is not a simulation of the thread itself, because the threads in Python still use the native threads of the operating system.

PyThreadState is only an abstraction of the thread state. However, in most sections of this book, for the sake of convenience, we are not strictly divided into threads and thread states. Therefore, in the future, we sometimes call PyThreadState a thread object, or a thread state object. We will strictly distinguish the two when analyzing the multi-threaded mechanism. The PyInterpreterState object mentioned below has similar considerations.

As mentioned earlier, in Win32, a thread cannot survive independently. It must survive in the process environment, and multiple threads can share some resources of the process. This is also true in Python. If there are two threads in the Python program and the same action is performed-import sys, how many copies of the sys module should this exist?

Is global shared or does each thread have a sys module? If each thread has its own independent module set, the memory consumption of Python will be astonishing. In Python, these modules are shared globally, as if they are all shared resources in the process. For the abstract concept of the Process, Python is implemented using the PyInterpreterState object.

In Win32, there are usually multiple processes, and Python can actually have multiple logical interpreters. In general, Python only has one interpreter. This interpreter maintains one or more PyThreadState objects. The threads corresponding to these PyThreadState objects use one bytecode execution engine in turn. Check whether it is very similar to the program execution model on a real machine?

When talking about multithreading, we can't help but talk about thread synchronization. In a Python virtual machine, a global Interpreter locks GILGlobal Interpreter Lock to implement thread synchronization. For details about this part, we will repeat the analysis of the Python multithreading mechanism.

Now we will discuss the two key objects mentioned above: PyInterpreterObject that represents the process concept and PyThreadState object that represents the thread concept.

1. Introduction to Python system files
2. How to correctly use Python Functions
3. Detailed introduction and analysis of Python build tools
4. Advantages of Python in PythonAndroid
5. How to Use the Python module to parse the configuration file?