Python script Performance Analysis
###################
# Python script Performance Analysis
###################
CProfile/profile/hotshot is used to collect statistics on the execution frequency and time consumption of each part of the Python script. pstats can be used to format the statistics.
CProfile is a C extension with low overhead. It is suitable for analyzing long-running Python programs. This module is recommended.
Profile, pure Python module, with significant overhead, but it is relatively easy to extend it
Hotshot, an experimental C module that focuses on minimizing overhead. It is no longer maintained and may be removed from Python in the future.
Profile and cProfile interfaces are the same, and cProfile is relatively new and may be unavailable on some platforms.
Use cProfile as an example to describe the Python script Performance Analysis Method
* Use the following command line:
$ Python-m cProfile [-o output_file] [-s sort_order] myscript. py
For example:
$ Python-m cProfile-o myscript. out myscript. py
Then use pstats to format the result:
$ Python-c import pstats; p = pstats. Stats ('myscript. out'); p. print_stats ()
You can specify the sorting field during formatting:
$ Python-c import pstats; p = pstats. Stats ('myscript. out'); p. sort_stats ('time'). print_stats ()
* Used directly inside the script:
Import cProfile
Import re
CProfile. run ('re. compile (foo | bar) ', 'restats ')
Import pstats
P = pstats. Stats ('restats ')
# Strip_dirs () Remove the path information before the module name. sort_stats (-1) are sorted by standard name (module/line/name), and print_stats prints the statistics.
P. strip_dirs (). sort_stats (-1). print_stats ()
# Sort by time and display the first 10 rows
P. sort_stats ('time'). print_stats (10)
# Display only the statistics related to the class init method by file
P. sort_stats ('file'). print_stats ('_ init __')
# First sort by time and then sort by cum, only 50% is output, and then only the part containing init is listed
P. sort_stats ('time', 'cum '). print_stats (. 5, 'init ')
# If you want to know who has called the above function, you can use
P. print_callers (. 5, 'init ')
* CProfile module description
Function:
CProfile. run (command, filename = None, sort =-1)
CProfile. runctx (command, globals, locals, filename = None) Success )¶
Class:
CProfile. Profile (timer = None, timeunit = 0.0, subcils = True, builtins = True)
Methods In the cProfile. Profile class:
Enable ()
Start collecting profiling data.
Disable ()
Stop collecting profiling data.
Create_stats ()
Stop collecting profiling data and record the results internally as the current profile.
Print_stats (sort =-1)
Create a Stats object based on the current profile and print the results to stdout.
Dump_stats (filename)
Write the results of the current profile to filename.
Run (cmd)
Profile the cmd via exec ().
Runctx (cmd, globals, locals)
Profile the cmd via exec () with the specified global and local environment.
Runcall (func, * args, ** kwargs)
Profile func (* args, ** kwargs)
* Stats class (pstats. Stats) Description
Strip_dirs () is used to remove the path information before the file name.
Add (filename, […]) Add the output file of profile to the Stats instance for Statistics
Dump_stats (filename) saves the Stats statistical result to the file
Sort_stats (key, […]) The most important function is used to sort the output of a profile.
Reverse_order () sorts the data in the Stats instance in reverse order.
Print_stats ([restriction,…]) Output the Stats report to stdout
Print_callers ([restriction,…]) Outputs Information about the function that calls the specified function.
Print_callees ([restriction,…]) Outputs Information about the function called by the specified function.
Sort_stats supports the following parameters:
Parameter description
'Calles' call count
'Cumulative time
'Cumtime' cumulative time
'File' file name
'Filename' file name
'Module' file name
'Ncall' call count
'Pcall' primitive call count
'Line' line number
'Name' function name
'Nfl 'name/file/line
'Stdname' standard name
'Time' internal time
'Tottime' internal time
* A typical output result:
197 function cballs (192 primitive cballs) in 0.002 seconds
Ordered by: standard name
Ncalltottime percall cumtime percall filename: lineno (function)
1 0.000 0.000 0.001 0.001 : 1 ( )
1 0.000 0.000 0.001 0.001 re. py: 212 (compile)
1 0.000 0.000 0.001 0.001 re. py: 268 (_ compile)
1 0.000 0.000 0.000 0.000 sre_compile.py: 172 (_ compile_charset)
1 0.000 0.000 0.000 0.000 sre_compile.py: 201 (_ optimize_charset)
4 0.000 0.000 0.000 0.000 sre_compile.py: 25 (_ identityfunction)
3/1 0.000 0.000 0.000 0.000 sre_compile.py: 33 (_ compile)
Output result description:
There are a total of 197 function calls, and the original call is 192. The original call does not contain recursive calls.
Sort by standard name. 3/1 indicates a recursive call. 1 indicates the number of original calls, and 3 indicates the number of recursive calls.
The number of times the ncils function is called.
Total running time of the tottime function, excluding the running time of the function called in the function
The average time of the percall function to run once, which is equal to tottime/ncils.
Cumtime total function running time, including the called function running time
The average time for the percall function to run once, which is equal
Filename: name of the lineno (function) function, the row number of the function, and the name of the function.
