Python source code analysis notes: Python compilation results

This article is senlie original, reproduced Please retain this address: http://blog.csdn.net/zhengsenlie

1. python Execution Process
1) Compile python source code to generate bytecode
2) Submit the compilation result to the python virtual machine. The virtual machine executes the bytecode one by one in sequence to generate the execution result.
Figure 7-1


2. Python compiler compilation result-PyCodeObject object
The compilation result of the Python compiler contains all useful static information such as strings, constant values, and bytecode in the source code.
During Python running, the static information is stored in the PyCodeObject object.
After running Python, the information will be stored in the pyc file.
The PyCodeObject object and the pyc file are two different forms of Python to compile the source file.


3. PyCodeObject in Python source code

/* Bytecode object */typedef struct {PyObject_HEAD int co_argcount;/* Number of location parameters */int co_nlocals;/* Number of local variables, including the number of location parameters */int co_stacksize; /* required stack space */int co_flags;/* CO _..., see below */PyObject * co_code;/* sequence of bytecode instructions. */PyObject * co_consts exists in PyStringObject form;/* PyTupleObject object, stores all constants */PyObject * co_names; /* PyTupleObject object, save all symbols */PyObject * co_varnames;/* set of local variable names */PyObject * co_freevars;/* implement things needed for closure */PyObject * co_cellvars; /* set of local variable names referenced by internal nested functions * // * The rest doesn 'tcount for hash/cmp */PyObject * co_filename;/* corresponding to The Code Block. the full path of the py file */PyObject * co_name;/* Code Block name, usually the name of the function or class */int co_firstlineno;/* Code Block. the start line of The py file */PyObject * co_lnotab;/* bytecode command and. the correspondence between source code row numbers in the py file. The equation of PyStringObject exists */void * co_zombieframe;/* for optimization only (see frameobject. c) */PyObject * co_weakreflist;/* to support weakrefs to code objects */} PyCodeObject;

Code Block: when a new namespace or scope is entered, a new Code Block is entered: when a new namespace or scope is entered, even if a new
Code Block. A Code Block corresponds to a PyCodeObject
The namespace is the context of the symbol. A namespace chain is composed of multiple namespaces.
In Python, modules, classes, and functions all correspond to an independent namespace.

# Three pycodeobjects are generated, which correspond to the entire file respectively. class A and def Funclass A: passdef Fun (): passa = A () Fun ()

3. pyc File
The pyc file contains three parts of independent information:
Magic number OF Python --> ensure Python compatibility
Pyc File Creation Time --> enables Python to automatically synchronize the pyc file with the latest py file
PyCodeObject object


Write the PyCodeObject object in the memory to the pyc file. The following functions are required:
W_byte
W_long
W_string


Pyexternal_writeobjecttofile calls w_object, and w_object traverses all fields in PyCodeObject,
Write these fields in sequence.
Write is eventually written in two forms: writing a number and writing a string

static void w_object(PyObject *v, WFILE *p){//……else if (PyCode_Check(v)){PyCodeObject *co = (PyCodeObject *)v;w_byte(TYPE_CODE, p); w_long(co->co_argcount, p);w_long(co->co_nlocals, p);w_long(co->co_stacksize, p);w_long(co->co_flags, p);w_object(co->co_code, p);w_object(co->co_consts, p);w_object(co->co_names, p);w_object(co->co_varnames, p);w_object(co->co_freevars, p);w_object(co->co_cellvars, p);w_object(co->co_filename, p);w_object(co->co_name, p);w_long(co->co_firstlineno, p);w_object(co->co_lnotab, p);}//…… }

Before writing an object, w_object will first write the TYPE_LIST, TYPE_CODE, or TYPE_INT identifier.
Re-loading of pyc files plays a vital role. If Python finds such an identifier in the pyc file, it indicates
The end of an object, the start of a new object, and the object type.


4. Write a string to the pyc File
Struct WFILE during write

Typedef struct {FILE * fp; PyObject * strings; // point to dict when writing, and point to list} WFILE when reading;

Strings in WFILE points to a PyDictObject object (PyStringObject, pymongobject) When marshal)

// W_object processing of strings else if (PyString_CheckExact (v) {if (p-> strings & PyString_CHECK_INTERNED (v) {// [1]: obtain the sequence number of the PyStringObject in strings. PyObject * o = PyDict_GetItem (p-> strings, v); // [2]: Non-first write of the intern string if (o) {long w = PyInt_AsLong (o); w_byte (TYPE_STRINGREF, p); w_long (w, p); goto exit;} // [3]: the first time the intern string is written to else {int OK; o = PyInt_FromSsize_t (PyDict_Size (p-> strings); OK = o & PyDict_SetItem (P-> strings, v, o)> = 0; Py_XDECREF (o); if (! OK) {p-> depth --; p-> error = WFERR_UNMARSHALLABLE; return;} w_byte (TYPE_INTERNED, p) ;}// [4]: write the common string else {w_byte (TYPE_STRING, p) ;}// write the string w_pstring (PyBytes_AS_STRING (v), PyString_GET_SIZE (v), p );}//...

How can I determine whether a string needs intern?
When writing a pyc file, strings is of the dict type.
Figure 7-6
When reading from the pyc file, strings is of the list type.
Figure 7-7


5. One PyCodeObject and multiple pycodeobjects
Co_consts in PyCodeObject is the hiding place of nested PyCodeObject
Figure 7-8

6. Python bytecode
104 byte codes

STOP_CODE() Indicates end-of-code to the compiler, not used by the interpreter.NOP() Do nothing code. Used as a placeholder by the bytecode optimizer.POP_TOP() Removes the top-of-stack (TOS) item.ROT_TWO() Swaps the two top-most stack items.ROT_THREE() Lifts second and third stack item one position up, moves top down to position three.ROT_FOUR() Lifts second, third and forth stack item one position up, moves top down to position four.//...