How to customize the search and physical implementation of the Python Module

In general, Python obtains a python module object from. py on the file system or the corresponding bytecode file. pyc or. pyo. Sometimes you need to customize this process. For example, when embedding Python as an application container, you want a special application packaging format, such as jar or war, or for some reason, you need to change the physical storage of the Python Module. For example, if the python module can be obtained from a key-value database, or the physical storage of the Python Module needs to be changed, it would be better to encrypt the source code (or pyc, and pyc is easily decompiled to py.

3.1 how to obtain a module object by programming

In [1]: import imp
 
In [3]: m = imp. new_module ("test ")
 
In [4]: code_obj = compile ("import OS", "test. py", 'exec ')
 
In [5]: code_obj
Out [5]: <code object <module> at 0x31b5830, file "test. py", line 1>
 
 
In [6]: exec code_obj in m. _ dict __
 
In [7]: m
Out [7]: <module 'test' (built-in)>
 
In [8]: dir (m)
Out [8]: ['_ builtins _', '_ doc _', '_ name _', '_ package __', 'OS']
 

Here, newmodule creates a corresponding module object, and the built-in function compile obtains the corresponding code object from a string (Source code), which can be exec.

3.2 How to customize Python Module search

Refer to the above link. The main point is that a class implementing Import Protocol can be used to load the hook module. This hook will be installed in sys. pathhooks.

This is an example of the import module from the Network (github:

3.3 How to customize Python Module Storage

The key to referring to the previous code is 'exec source in m. _ dict _ 'Here, source can be a piece of source code, such as "import OS" or a code object, which can be serialized or deserialized. In fact, pyc is the serialization of the object (added with the timestamp, magic number, and crc verification). For performance considerations, deserialization is slightly better than compiling the py source file, the serialized results of code objects can be stored. Get the code object marshal string from the storage device in Importer.

• Obtain its code object marshal string from a py file

Import marshal
Source = open ("test. py"). read ()
Co = compile (source, "test. py", 'exec ')
Co_s = marshal. dumps (co)

• Obtain a python module from a code object marshal string

Import marshal, imp
Def load_module (co_str ):
M = imp. new_module ("test ")
Co = marshal. loads (co_str)
Exec co in m. _ dict __
Return m

Author: MaTao <qingant@gmail.com>

Date: 2012-03-20 Tue

HTML generated by org-mode 6.33x in emacs 23