I want to know about the python performance debugging method. The result shows this article and I want to translate it into a record.
Original from: http://docs.python.org/library/profile.html? Highlight = profile # cProfile
1. Introduction to Performance Analyzer
Profiler is a program used to describe the program performance at runtime and provide statistical data for presentation from different aspects. Python contains three modules that provide such functions: cProfile, profile, and pstats. These analyzers provide deterministic Analysis of Python programs. A series of report generation tools are also provided, allowing you to quickly check the analysis results.
The Python Standard Library provides three different performance analyzers:
- CProfile is recommended to most users. It is an extension application of C. It is suitable for analysis and running for a long time because of its reasonable running overhead. Is based on lsprof.
- Profile, a pure python module with the same interface as cProfile. It increases the running overhead when analyzing programs. If you want to extend the profiler function, try to inherit this module.
- Hotshot, a experimental c Module, focuses on reducing the running overhead during analysis, but it is at the cost of a longer number of data post-processing times. However, this module is no longer maintained and may be discarded in the new python version.
2. Usage
Def foo (): sum = 0 for I in range (10000): sum + = I sumA = bar () sumB = bar () return sum def bar (): sum = 0 for I in range (100000): sum + = I return sum if _ name _ = "_ main __": import cProfile # print the analysis result directly to the console cProfile. run ("foo ()") # Save the analysis result to the file, but the content is not readable... call the pstats module to analyze the results cProfile. run ("foo ()", "result") # You can also directly use the command line for operations #> python-m cProfile myscript. py-o result import pstats # create Stats object p = pstats. stats ("result") # The effect of this line and directly run cProfile. run ("foo ()") shows the same effect as p. strip_dirs (). sort_stats (-1 ). print_stats () # strip_dirs (): Remove irrelevant path information from all module names # sort_stats (): sort the printed information according to the standard module/name/line string # print_stats (): print all analysis information # Sort the information by function name p. strip_dirs (). sort_stats ("name "). print_stats () # sort by the accumulated running time in a function # print_stats (3): only the information of the first three rows of functions is printed, and the parameters can be decimals, indicates the function information of the first few percent p. strip_dirs (). sort_stats ("cumulative "). print_stats (3) # Another method is p. sort_stats ('time', 'cum '). print_stats (. 5, 'foo') # first sort by time, then sort by cumulative time, and then output the first 50% containing function information # If you want to know which functions have called bar, p. print_callers (0.5, "bar") # Similarly, check which functions p are called in the foo () function. print_callees ("foo ")
The above is a simple application of the profile and pstats modules.
3. Graphic analysis results
4. What is Deterministic Performance Analysis (Deterministic Profiling)
Deterministic performance analysis refers to the time taken to execute all function calls, returns, and exception events, and the time interval between them. In contrast, statistical performance analysis refers to sampling valid program instructions and then exporting the time required. The latter consumes less, but the results are not accurate enough.
In Python, because it is an explanatory language, the interpreter execution is added during program execution, and performance analysis is not required for this part of execution. Python automatically provides a hook for each event to locate the code to be analyzed. In addition, because the essence of the Python interpreted language often requires a lot of additional overhead when executing the program, and deterministic performance analysis only adds a little overhead for processing. In this way, deterministic performance analysis is not costly, but also provides a wealth of statistics.
The statistics on the number of function calls can be used to determine bugs in the program, for example, the number of times that do not conform to common sense, obviously more, and so on. It can also be used to determine possible inline functions. Statistics on the internal running time of the function can be used to determine "hot loops", which requires a long running time and needs to be optimized. Statistics on the accumulated time can be used to determine high-level errors, for example, an error occurs in algorithm selection. Python performance analysis allows direct comparison of Recursive Implementation and iterative Implementation of algorithms.