###################
#Python脚本性能剖析
###################
Cprofile/profile/hotshot is used to count statistics such as how often each part of a Python script runs and how long it takes. Pstats can be used to format this information
CProfile is a C extension. Low overhead, suitable for parsing long-executed Python programs, recommended for use with this module
Profile Pure Python module, there is significant overhead, but want to extend the relatively easy
Hotshot, experimental C module. Primary focus on overhead minimization, which is no longer maintained in the future may be removed from Python
The profile and Cprofile interfaces are the same. Cprofile relatively new may not be available on some platforms
Take Cprofile as an example to illustrate how Python scripting performance profiling methods
* Use in command line mode:
$ python-m cProfile [-O output_file] [-s sort_order] myscript.py
Like what:
$ python-m cprofile-o myscript.out myscript.py
The results are then formatted using pstats:
$ python-c "Import pstats; P=pstats. Stats (' myscript.out '); P.print_stats () "
Ability to specify sort fields when formatting:
$ python-c "Import pstats; P=pstats. Stats (' myscript.out '); P.sort_stats (' time '). Print_stats () "
* Used directly inside the script:
Import CProfile
Import re
Cprofile.run (' Re.compile ("Foo|bar") ', ' restats ')
Import pstats
p = pstats. Stats (' Restats ')
#strip_dirs () Remove the path information before the module name, Sort_stats (-1) sorted by standard name (module/line/name), print_stats print statistics
P.strip_dirs (). Sort_stats ( -1). Print_stats ()
#按time排序并显示前10行
P.sort_stats (' time '). Print_stats (10)
#按file排序仅仅显示class statistics about the Init method
P.sort_stats (' file '). Print_stats (' __init__ ')
#先按time排序再按cum排序, just output 50%. Then only the parts that include Init are listed
P.sort_stats (' time ', ' cum '). Print_stats (. 5, ' init ')
#若想知道谁调用了上述函数能够使用
P.print_callers (. 5, ' init ')
*cprofile Module Description
Function:
Cprofile.run (Command, Filename=none, Sort=-1)
Cprofile.runctx (Command, Globals, locals, Filename=none)?
Class:
Cprofile.profile (Timer=none, timeunit=0.0, Subcalls=true, Builtins=true)
Methods under the Cprofile.profile class:
Enable ()
Start collecting profiling data.
Disable ()
Stop collecting profiling data.
Create_stats ()
Stop collecting profiling data and record the results internally as the current profile.
Print_stats (Sort=-1)
Create a Stats object based on the current profiles and print the results to stdout.
Dump_stats (filename)
Write the results of the current profiles to filename.
Run (CMD)
Profile the cmd via exec ().
Runctx (CMD, globals, locals)
The profile of the CMD via exec () with the specified global and local environment.
Runcall (func, *args, **kwargs)
Profile func (*args, **kwargs)
*stats Class (Pstats.stats) description
Strip_dirs () The path information before the file name is removed.
Add (filename,[...]) Add profile output file to stats instance statistics
Dump_stats (filename) Save the stats statistics to a file
Sort_stats (key,[...]) The most important function to sort the output of a profile
Reverse_order () Rearrange the data in the stats instance in reverse order
Print_stats ([restriction,...]) Export the stats report to stdout
Print_callers ([restriction,...]) Outputs information about the function that called the specified function
Print_callees ([restriction,...]) Outputs information about functions that have been called by the specified function
The following parameters are supported by the Sort_stats:
The meaning of the parameters
' Calls ' call Count
' Cumulative ' cumulative time
' Cumtime ' cumulative time
' File ' file name
' filename ' file name
' Module ' file name
' Ncalls ' call Count
' Pcalls ' Primitive Call Count
' Line ' line number
' Name ' function name
' NFL ' name/file/line
' Stdname ' standard name
' Time ' internal time
' Tottime ' internal time
* A comparison of typical output results:
197 function calls (192 primitive calls) in 0.002 seconds
Ordered By:standard Name
Ncalls tottime percall cumtime percall filename:lineno (function)
1 0.000 0.000 0.001 0.001 <string>:1 (<module>)
1 0.000 0.000 0.001 0.001 re.py:212 (compile)
1 0.000 0.000 0.001 0.001 re.py:268 (_compile)
1 0.000 0.000 0.000 0.000 sre_compile.py:172 (_compile_charset)
1 0.000 0.000 0.000 0.000 sre_compile.py:201 (_optimize_charset)
4 0.000 0.000 0.000 0.000 sre_compile.py:25 (_identityfunction)
3/1 0.000 0.000 0.000 0.000 sre_compile.py:33 (_compile)
Output result Description:
There is a common 197 function call, the original call is 192 times, and the original invocation description does not include a recursive call.
Sort by standard name.
3/1 indicates a recursive call has occurred, 1 is the original number of calls, and 3 is the number of recursive calls
The number of times the Ncalls function was called
Tottime The total execution time of the function. Remove function execution time called in function
The average time that the Percall function executes once equals tottime/ncalls
Cumtime The total execution time of the function. function execution time with call
The average time that the Percall function executes once. equals Cumtime/ncalls
The name of the file in which the Filename:lineno (function) function is located, and the line number of the function. Name of function
References:
Https://docs.python.org/2/library/profile.html
Python Scripting Performance Profiling
