Python performance optimization skills summary, python performance optimization skills

Python performance optimization skills summary, python performance optimization skills

1. using measurement tools to quantify performance can improve performance. Commonly UsedtimeitAndmemory_profilerIn additionprofile,cProfile,hotshot,memory_profilerUsedpsutil, So it cannot be tracked.cpython;

2. Use C to solve time-consuming processing. c is synonymous with efficiency. It is also the main way for python to solve efficiency problems. Sometimes I think python is the perfect partner of c. Commonly used isCythonDirectly convert the py code to c and then use it like using the py package. The second isctypes, The most efficient, and finallyCPythonAndcffiThey all exist;

3. optimization Algorithms, all common language faults, and improvement of algorithms I think are above all improvement, but they are also the most difficult. Fortunately, most of the commonly used algorithms are already packed, unless you dig yourself into the trap, therefore, it is important to understand how data structures and common APIs in the standard library are implemented;

Some implementations in section 4.2 have made more efficient packages to replace some common implementations in python. If the bottleneck isstringio、pickle、profileIn this type, you can consider replacing it with the c version;

5. Use tuple as much as possible in the data structure, especially when there is a large amount of data, it is not enough to list or use tuple as much as possible.classYou can addslotIf the efficiency is not enough, it can only be combined with 2 to accelerate;

6. Delayed loading,importIt is not necessary to write a page at the beginning, where it can be, the more fragmented the more the package loading delay or even not loaded;

7. UsemultiprocessingTo implement multiple threads, you can jump out of the GIL limit;

8. the python processing cycle is very bad, and the explanatory language is like this. Compared with other compiled languages, it can significantly improve the performance by reducing the number of loops and nesting times, of course, using pypy does not solve this problem;

9. using the accelerator, I like the use of psyco very much. If we use version 2.7-, it would be a lazy choice. Now we no longer need to maintain it. The Founder has gone to pypy, Which is Python implemented using python, c and ,. net, java intermediate language, the method is very clever, love, but the disadvantage is that the library support is not complete, my project can basically support, just solve a few small problems, if the performance bottleneck is in the loop and memory, you can try it. The biggest advantage is that you do not need to change the code or make other settings without any intrusion.

References:

Python code performance optimization tips: https://www.ibm.com/developerworks/cn/linux/l-cn-python-optim/

Python performance optimization tips: http://kuanghy.github.io/2016/09/26/python-optimize