Python code object and PYc file (ii)

Tag: port \ n Instruction class open path LIS tuple value

Previous section: Python code object with PYc file (i)

Specific procedures for creating PYC files

As we mentioned earlier, Python will dynamically load the module with import or from xxx import xxx, and if no corresponding PYC or DLL file is found, the PYc file will be created on the basis of the Py file, as previously stated, The Pycodeobject object is stored in the PYc file, then we need to figure out how Pycodeobject writes to the PYc file.

Import.c

Static Voidwrite_compiled_module (Pycodeobject *co, Char *cpathname, time_t mtime) {file *fp;//exclusive open file FP = open_exclusive (Cpathname);//<1> written to Python's Magic numberpymarshal_writelongtofile (Pyc_magic, FP, py_marshal_version);//<2 > Write Pycodeobject Object Pymarshal_writeobjecttofile ((Pyobject *) CO, FP, Py_marshal_version);//<3> Write Time Information Pymarshal_writelongtofile ((long) mtime, FP, py_marshal_version); Fflush (FP); fclose (FP);}

　　

The code in Write_compiled_module is slightly reduced, and we only keep the ones that need the most attention. It can be found that a PYc file actually contains 3 pieces of independent information, the magic number in Python, the Pycodeobject object, and the time when the PYc file was created

At <1>, Python writes the value pyc_magic to the beginning of the file, Pyc_magic is an integer value, and different versions of Python define different magic numbers, and when Python loads a pyc file, Checks whether the pyc_magic in the PYc file corresponds to the pyc_magic of the current Python version and avoids Python2.5 loading Python1.5 files that are compiled. The reason for this check is that different versions of Python's bytecode instructions may have different changes, some of the old instructions will be replaced by new instructions, and even add new instructions, which leads to the incompatibility of Python issues

In import.c, you can find all versions of magic number Python1.5 to Python2.5 in the comments in the source code, and we can look at the magic number defined by Python2.5:

Import.c

#define MAGIC (62131 | ((long) ' \ R ' <<16) | ((long) ' \ n ' <<24)) static Long pyc_magic = Magic;

　　

In PYc, the action to write time information to the PYc file is completed at <3>. Including the time information in the PYc file allows Python to compare pyc with the latest py file, and if it is found that the PYC generation time is earlier than the modification time of the Py file, the Py file is modified and the PYc file is recompiled

At <2> in the code above, Python calls the Pymarshal_writeobjecttofile method, writes the in-memory Pycodeobject object to the PYc file, and in Write_compiled_module, The action to write data to a PYC file is eventually concentrated in several functions shown below

Now, let's take a look at the Pymarshal_writeobjecttofile method.

Marshal.c

void Pymarshal_writeobjecttofile (Pyobject *x, FILE *fp, int version) {Wfile WF;WF.FP = Fp;wf.error = 0;wf.depth = 0;wf.stri NGS = (Version > 0)? Pydict_new (): Null;wf.version = Version;w_object (x, &WF); Py_xdecref (wf.strings);}

　　

Pymarshal_writeobjecttofile This method calls W_object this method to actually write the object to the file

Marshal.c

static void W_object (Pyobject *v, Wfile *p) {... else if (Pytuple_check (v)) {...} else if (Pylist_check (v)) {...} else if (Pydict_check (v)) {...} ... else if (Pycode_check (v)) {Pycodeobject *co = (Pycodeobject *) v;w_byte (Type_code, p); W_long (Co->co_argcount, p); w_ Long (co->co_nlocals, p); W_long (Co->co_stacksize, p); W_long (Co->co_flags, p); W_object (Co->co_code, p); W _object (Co->co_consts, p); W_object (Co->co_names, p); W_object (Co->co_varnames, p); W_object (co->co_ Freevars, p); W_object (Co->co_cellvars, p); W_object (Co->co_filename, p); W_object (Co->co_name, p); W_long ( Co->co_firstlineno, p); W_object (Co->co_lnotab, p);} ......}

　　

From the above code we can see that in W_object, each domain in the pycodeobject is traversed, and these fields are written one at a time.

What happens when W_object faces a Pylistobject object?

Marshal.c

else if (Pylist_check (v)) {w_byte (type_list, p); n = pylist_get_size (v); W_long ((Long) n, p); for (i = 0; i < n; i++) {W_ob Ject (Pylist_get_item (v, i), p);}}

　　

As in front of Pycodeobject, w_object or traversal, writes each element of the Pylistobject object to the PYc file once

We'll go through the w_object. This method will find that before writing to any object, a type identity such as Type_list or Type_code is written, which is essential for the PYc file to be loaded again. If we were simply writing numeric and string information from the object to the PYc file, it would be difficult for us to restore these values or strings to the objects previously in memory if there were no corresponding type information. When Python loads the PYc file, it discovers the type information, which indicates the end of the previous object, the beginning of the new object, and the type of object that the new object is. This way, when Python loads the PYc file, the loader will know when to load the operation

The definition of a type identity in Python:

Marshal.c

#define TYPE_NULL ' 0 ' #define Type_none ' N ' #define TYPE_FALSE ' F ' #define type_true ' T ' #define Type_stopiter ' S ' #define Type_ellipsis   '. ' #define Type_int ' i ' #define type_int64 ' i ' #define type_float ' F ' #define type_binary_float ' g ' #define Type_complex ' x ' # Define Type_binary_complex ' y ' #define Type_long ' l ' #define type_string ' s ' #define type_interned ' t ' #define Type_ Stringref ' R ' #define type_tuple ' (' #define type_list ' [' #define TYPE_DICT ' {' #define Type_code ' C ' #define Type_unicode ' u ' #define Type_unknown '? ' #define Type_set ' < ' #define Type_frozenset  ' > '

　　

Python code object and PYc file (ii)