Python code streamlining and Optimization

Python code streamlining and Optimization

Python is very simple, easy to use, highly efficient development, good portability, and rich code resources. It is widely used. However, the dynamic library compiled by the Python code is relatively large, and the python library is comprehensive. The disadvantage is that the Library is relatively large.

The memory usage method doubles with the introduction of the py library. Here we will discuss how to slim down Python and how to optimize the memory usage.

I. How to slim down the Python dynamic library.

Python code is still very refined, so it is difficult to reduce the size of small code, but there are still some ideas to reduce the size of the Python library.

1. strip python dynamic library.

Dynamic libraries generally contain symbol tables, which are useful for calling. However, for the release version, you can call the symbol table by using the strip command, in this way, the size can be reduced from 8 to 9 MB to less than 3 MB.

2. Use the code optimization option-O3 to optimize the code to the maximum extent, including optimizing the size of the generated binary code. The disadvantage is that debugging is difficult after optimization.

3. Remove Doc String from the code.

In Python code, the help description of the module defined by the PyDoc_STRVAR macro can be removed by specifying -- without-doc-strings during configure, so that the generated pyconfig. will there be the following definition in h:

# Define WITH_DOC_STRINGS 1

This can reduce the size of the generated dynamic library. Of course, it can also reduce the memory usage of the module during runtime, because these modules no longer contain help information.

4. Remove unicode support.

Unicode support is not required in python. Of course, python 3 is another matter. In python, unicode can be encoded in UTF-8 format. The following parameters can be used when configure is removed from unicode:

-- Enable-unicode = no

In this way, the following definition is removed in pyconfig. h:

# Define Py_USING_UNICODE 1

Ii. How to reduce the size of the Python extension library?

The Python extension library is placed in the lib directory. You can execute the following command in the lib directory to compile the Python code:

Python-OO-m compileall.

This will generate a library file with the pyo extension, and the-OO parameter will remove the doc string, so that when there are many annotations in The py file, the size of the compilation target file will be significantly reduced.

Do not use absolute path:

For example, python-OO-m compileall/path/to/python/lib uses the absolute path command, because when the pyo file is generated ,, each function and class method will generate a code object one by one, and each code object will save the path of its module. If the absolute path is used, when the path is long, when there are many functions, the pyo file size is greatly increased.

Of course, you can also reduce the memory usage during code execution.

3. How to cut down the extended library.

There is a py2exe tool that can package python code and dependent dynamic libraries and package python's necessary extension libraries into a zip file. But in fact, this zip package is often not the most streamlined. In fact, the biggest difficulty in cutting is to find all the dependent modules. You can use the following method to find other modules that a module depends on:

import importlibdef module_diff(mod):    import sys    keys = []    for key in sys.modules.keys():        keys.append(key)    importlib.import_module(mod)    for key in sys.modules.keys():        if not key in keys:            print key,sys.modules[key]

To view the modules that the multiprocessing module depends on, run the following command:

module_diff('multiprocessing')

The following output is displayed:

Multiprocessing. atexit Nonemultiprocessing. weakref Nonemultiprocessing. signal Nonethreading
 
  
CPickle
  
   
_ Multiprocessing
   
    
Multiprocessing. OS Nonemultiprocessing. itertools Nonemultiprocessing. threading Nonemultiprocessing. util
    
     
Multiprocessing. sys NonecStringIO
     
      
Multiprocessing. _ multiprocessing Nonemultiprocessing. multiprocessing Nonethread
      
        Atexit
       
         Multiprocessing
        
          Weakref
         
           Itertools
          
            Time
           
             Multiprocessing. process
            
             

             

In this way, you can know the dependent modules.
             
To view all modules, it is simpler:
             

             
def print_all_module():    import sys    keys = []    for key in sys.modules.keys():        print key,sys.modules[key]
             
After the code Initialization is complete, execute the above function to know the modules required for running the program.