function mechanism of Python virtual machine (i.)

Pyfunctionobject Object

In Python, anything is an object, and the function is no exception. The abstract mechanism of the function is implemented by a Python object--pyfunctionobject.

typedef struct {    pyobject_head    pyobject *func_code;//compiled Pycodeobject object    pyobject *func_globals;// The global namespace    pyobject *func_defaults;//default parameter (tupple or NULL)    pyobject *func_closure;//null or a tuple of cells when the function is run objects, document for implementing closure (closure)    pyobject *func_doc;//function (pystringobject)    pyobject *func_name;//function name, function __name_ _ Property, (Pystringobject)    Pyobject The __dict__ property of the *func_dict;//function (pydictobject or null)    pyobject *func_weakreflist;    Pyobject the __module__ of the *func_module;//function, which can be any object} Pyfunctionobject;

　　

In Python, there are two objects and functions related to Pycodeobject and Pyfunctionobject. The Pycodeobject object is a static representation of a Python source code that, after compiling, generates one and only one Pycodeobject object for a code block. This Pycodeobject object contains some static information in this code block, the so-called static information can be seen in the source code information, such as code block has a = 1 expression, then the symbol A and the value 1, and the connection between them is a kind of static information, This information is stored in Pycodeobject's constant table co_consts, symbol table Co_names, and bytecode Co_code, which are available at compile time because Pycodeobject is a compile-time result

Unlike Pyfunctionobject, the Pyfunctionobject object is generated dynamically by Python code at run time, and more precisely when a DEF statement is executed. In Pyfunctionobject, the static information for this function is also included, which is stored in the Func_code. In fact, Func_code is bound to point to the Pycodeobject object that corresponds to the function code. In addition, the Pyfunctionobject object also contains the dynamic information that some functions must perform, i.e. contextual information, such as Func_globals, which is the global scope that the function is associated with at execution time. The relationship between the symbol and value that is used by global has to be determined at run time, so this part must be created dynamically at run time and cannot be stored in Pycodeobject

For a piece of Python code, its corresponding Pycodeobject object has only one, and the code corresponding to the Pyfunctionobject object may have multiple, such as a function multiple calls, Python creates multiple Pyfunctionobject objects at run time, and each func_code field of a Pyfunctionobject object is associated to this Pycodeobject object. 1-1 shows the relationship between Pyfunctionobject and Pycodeobject objects:

Figure 1-1

No parameter calls

Our anatomy of a Python function starts with a parameterless call, because a parameterless call is the simplest form

# cat Demo.pydef F ():    print ("Function") f () # python2.5......>>> Source = open ("demo.py"). Read () >>> Co = Compile (source, "demo.py", "exec") >>> import dis>>> Dis.dis (CO)  1           0 load_const               0 (< Code object f at 0x7fd9831c3648, file "demo.py", line 1>)              3 make_function            0              6 store_name               0 (f)  5
   9 load_name                0 (f)             call_function            0             pop_top                          load_const               1 (None)             19 Return_value        >>> from demo import ffunction>>> Dis.dis (f)  2           0 load_const               1 (' Function ')              3 print_item                        4 print_newline                     5 load_const               0 (None)              

　　

Demo.py generates two Pycodeobject objects, one is the Pycodeobject object generated after the source file is compiled, and the Pycodeobject object generated after the function f is compiled, contained in the Pycodeobject object corresponding to the source file.

As can be seen from the above example, when dis compiles demo.py, the bytecode of the F function is not compiled, and when we pass the function f to the DIS module, we get the byte code corresponding to the function F. This indicates that the byte code of function f is not in the Co_code domain of the Pycodeobject object corresponding to demo.py, but in the Pycodeobject object corresponding to function f, the Pycodeobject object corresponding to function f, Contained in the Pycodeobject object corresponding to the demo.py, this is the hierarchy of bytecode directives that we must pay attention to. Thus, the declaration of a function is actually separated from the implementation of the function, and they are separated in different Pycodeobject objects. Although from the source code, our first instinct is that the declaration and implementation should be together, but actually not.

When a Python virtual machine executes a DEF statement, it dynamically creates a function, the Pyfunctionobject, in which the make_function instruction is a key:

Ceval.c

Case make_function:v = POP (); Obtain the PYCODEOBJECTX = Pyfunction_new (V, f->f_globals) corresponding to the function f; Py_decref (v);//Handle the default value of the function parameter if (x! = NULL && oparg > 0) {v = pytuple_new (oparg); if (v = = null) {py_decref (x); x = NUL L;break;} while (--oparg >= 0) {w = POP (); Pytuple_set_item (V, Oparg, W);} Err = pyfunction_setdefaults (x, v); Py_decref (v);} PUSH (x); break;

　　

Before Make_function, the Python virtual opportunity executes "0 load_const 0", pressing the Pycodeobject object corresponding to function f into the runtime stack. When the make_function is executed, it is then taken out and then the F_globals object is a parameter with the global namespace maintained in the object and the current Pyframeobject object, by Pyfunction_ The new Pyfunctionobject object is created, and this f_globals will become the global namespace when the function F runtime stack

Funcobject.c

Pyobject * Pyfunction_new (pyobject *code, Pyobject *globals) {//[1]: Required memory space for Pyfunctionobject object Pyfunctionobject *op = Pyobject_gc_new (Pyfunctionobject, &pyfunction_type); static Pyobject *__name__ = 0;if (op! = NULL) {...// Set Pycodeobject Object Op->func_code = code;//Set global namespace op->func_globals = globals;//Set function name Op->func_name = (( Pycodeobject *) code)->co_name; Py_incref (op->func_name); op->func_defaults = NULL; /* No default arguments */op->func_closure = constant Object table in null;//function consts = ((Pycodeobject *) code)->co_consts;//function document if ( Pytuple_size (consts) >= 1) {doc = Pytuple_getitem (consts, 0); Pystring_check (DOC) &&! Pyunicode_check (doc)) doc = Py_none;} Elsedoc = Py_none, ...} Elsereturn null;_pyobject_gc_track (OP); return (Pyobject *) op;}

　　

After creating the Pyfunctionobject object, Make_function will also perform some action on the function parameters, because our f is the simplest parameterless function, without any parameters, so the transfer mechanism of the parameters will be parsed in the following chapters.

After the make_function is finished, the newly created Pyfunctionobject object is pressed into the runtime stack with push operations, and the subsequent "6 Store_name 0" and "9 Load_name 0" are no longer detailed. Below, figure 1-1 shows the runtime stack and the local namespace when the Def F () statement finishes executing:

Figure 1-1 Virtual machine state after creating a Pyfunctionobject object

Function call

Starting with "Call_function 0", the Python virtual machine begins to invoke the function.

Ceval.c

Case Call_function:{pyobject **SP; Pcall (pcall_all); sp = Stack_pointer;x = Call_function (&SP, oparg); stack_pointer = SP; PUSH (x); if (x! = NULL) Continue;break;}

　　

The implementation of the call_function is very simple, just get the stack-top pointer and then call the Call_function function. So we're going to follow the steps of the Python virtual machine into the call_function function.

Ceval.c

Static Pyobject * Call_function (pyobject ***pp_stack, int oparg) {//[1]: processing function parameter information int na = Oparg & 0xff;int nk = (Oparg & Gt;> 8) & 0xff;int n = Na + 2 * nk;//[2]: Get Pyfunctionobject object Pyobject **pfunc = (*pp_stack)-n-1; Pyobject *func = *pfunc; Pyobject *x, *w;if (Pycfunction_check (func) && nk = = 0) {...} Else{if (Pymethod_check (func) && pymethod_get_self (func)! = NULL) {...} Elsepy_incref (func); Read_timestamp (*PINTR0);//[3]: Call to Pyfunctionobject object if (Pyfunction_check (func)) x = Fast_function (func, Pp_stack, n , Na, nk); elsex = Do_call (func, Pp_stack, Na, NK), ...} ... return x;}

　　

It can be seen that not only function, but also cfunction and method will enter the Call_function, where cfunction is an intrinsic function or class method, method is an example. Here, our focus is on function, regardless of cfunction and method.

In the call_function [1], there are some actions to handle the function parameter information, n indicates how many elements of the stack at the top of the heap are related to the parameters in the runtime stack. Of course, for our F, where Na, NK and N are all 0, so [2] Pfunc is (*pp_stack)-1, because Pp_stack is the top pointer of the current runtime stack that we passed in the call_function instruction code

Obviously, *pfunc, or func, points to the Python virtual machine that was pushed into the runtime stack by the "9 Load_name 0" instruction before call_function in Make_ The Pyfunctionobject object created in the function

Python Virtual machine code is checked at [3] and will enter Fast_function. Here we also enter the Fast_function function

Ceval.c

Static Pyobject * Fast_function (pyobject *func, pyobject ***pp_stack, int n, int na, int nk) {Pycodeobject *co = (pycodeobj ECT *) Pyfunction_get_code (func); Pyobject *globals = Pyfunction_get_globals (func); Pyobject *argdefs = Pyfunction_get_defaults (func); Pyobject **d = null;int nd = 0;......//[1]: Fast Channel if general function (argdefs = = NULL && Co->co_argcount = = N && nk = = 0 &&co->co_flags = = (Co_optimized | Co_newlocals | Co_nofree)) {Pyframeobject *f; Pyobject *retval = NULL; Pythreadstate *tstate = Pythreadstate_get (); Pyobject **fastlocals, **stack;int i;......f = Pyframe_new (Tstate, CO, globals, NULL);.//Enter the Python virtual machine again retval = Pyeval_ Evalframeex (f, 0);...... return retval;} if (argdefs! = NULL) {d = &pytuple_get_item (argdefs, 0); nd = ((Pytupleobject *) argdefs)->ob_size;} The implementation of Pyeval_evalcodeex will still call Pyeval_evalframeexreturn Pyeval_evalcodeex (CO, globals, (Pyobject *) NULL, (*pp_stack)- N, Na, (*pp_stack)-2 * nk, NK, D, ND, Pyfunction_get_closure (func));

　　

After entering Fast_function, the first Pyfunctionobject objects are extracted from the Pycodeobject object and the information such as the global namespace when the function is run. Then we can see that the execution of Fast_function is divided into two paths, the parameterless function will be judged at fast_function [1], enter the fast channel of the function, and the Python virtual opportunity creates a new Pyframeobject object for it. And then call Pyeval_evalframeex.

In the other way, it is Pyeval_evalcodeex, the function Pyeval_evalcodeex contains a series of complex actions, but if the trace continues, it will still be called Pyeval_evalframeex function to complete the execution of bytecode instructions. Starting with Pyeval_evalframeex, the Python virtual machine actually begins to enter the state of the function call. This process is essentially a simulation of a function call on the x86 platform, creating a new stack frame when the function is called, and executing the function code in the new stack frame.

It is important to note that the impact of the Pyfunctionobject object has disappeared in the final passing of Pyeval_evalframeex. The real impact on the stack frame is the Pycodeobject object and the global namespace stored in the Pyfunctionobject. Plainly, Pyfunctionobject only transports Pycodeobject objects and global namespaces for calling Pyeval_evalframeex

Before we mentioned the fast channel of the function, what kind of function can enter the fast channel? Functions such as C, C + +, and Java, which are functions such as func (x, y), are not possible if functions such as Func (A, *args, **kwargs) are unique to Python. Python depends on the function parameter to determine if it can enter the fast-track

function mechanism of Python virtual machine (i.)