Python Performance Optimization Techniques Summary _python

1. The use of measuring tools, quantitative performance can improve performance, commonly used timeit and memory_profiler , in addition to, profile cProfile , hotshot etc., memory_profiler used psutil , so can not track cpython the expansion;

2. c to solve time-consuming processing, c is synonymous with efficiency, and Python is the main way to solve the problem of efficiency, and even sometimes I think Python is the perfect partner C. It is commonly used Cython , directly to the PY code C and then can be like using the same as the Py bag, followed by ctypes the most efficient existence, CPython and finally and cffi all the existence of cock cock;

3. Optimization algorithm, all language problems, the improvement of the algorithm I think is above all ascension, but also the most difficult, fortunately now most of the commonly used algorithms have been sealed, unless they dig their own pits, so understand the standard library of data structure and common API is how to achieve very important;

4.2 Implementation Someone has done a more efficient package to replace some of the common implementations in Python, if the bottleneck on stringio、pickle、profile this class can be considered to replace the version of C;

5. Data structure as far as possible using the tuple tuple, especially when the volume of data, it is not a list can also, try not class to use, if you must use can add slot , efficiency is not enough can only be combined with the acceleration;

6. Delay loading, import not necessarily to write at the beginning of a page, where all can be, the more fragments of the package can be loaded delay even not loaded;

7. Use multiprocessing to achieve multithreading, you can jump out of the Gil limit;

8.python processing cycle is very bad, interpretive language is like this, with other compiler language than is a snail, so reduce the number of cycles and nesting times can significantly improve performance, of course, the use of pypy is not the problem;

9. Using accelerators, like the use of Psyco, if the 2.7-version is a lazy person's choice, now no longer maintained, the founder went to the pypy,pypy is Python implemented Python, the bottom to platform-dependent C,. NET, Java in the middle language, the way is very clever, big love, but the disadvantage is that the library's support is not perfect, my project can basically support, solve a few small problems, if the performance bottleneck in the loop and memory can try, the biggest advantage is not need to change a code and do another set, without any intrusion.

Resources:

Python Code Performance Optimization tips: https://www.ibm.com/developerworks/cn/linux/l-cn-python-optim/

Python performance optimization tips: http://kuanghy.github.io/2016/09/26/python-optimize