###################
#Python脚本性能剖析
###################
Cprofile/profile/hotshot is used to count statistics on the frequency and time of execution of each part of a Python script, pstats can be used to format this information
CProfile, which is a C extension, has a small overhead and is suitable for profiling a long-running Python program, recommended for use with this module
Profile, a pure Python module, has significant overhead, but it's relatively easy to extend it.
Hotshot, the experimental C module, focuses on minimizing overhead and is no longer maintained in the future and may be removed from Python
The profile and Cprofile interfaces are the same, cprofile are relatively new and may not be available on some platforms
Take Cprofile as an example to illustrate how Python scripting performance profiling methods
* Use in command line mode:
$ python-m cProfile [-O output_file] [-s sort_order] myscript.py
For example:
$ python-m cprofile-o myscript.out myscript.py
The results are then formatted using pstats:
$ python-c "Import pstats; P=pstats. Stats (' myscript.out '); P.print_stats () "
You can specify a sort field when formatting:
$ python-c "Import pstats; P=pstats. Stats (' myscript.out '); P.sort_stats (' time '). Print_stats () "
* Used directly inside the script:
Import CProfile
Import re
Cprofile.run (' Re.compile ("Foo|bar") ', ' restats ')
Import pstats
p = pstats. Stats (' Restats ')
#strip_dirs () Remove the path information before the module name, Sort_stats (-1) sorted by standard name (module/line/name), print_stats print statistics
P.strip_dirs (). Sort_stats ( -1). Print_stats ()
#按time排序并显示前10行
P.sort_stats (' time '). Print_stats (10)
#按file排序只显示class statistics about the Init method
P.sort_stats (' file '). Print_stats (' __init__ ')
#先按time排序再按cum排序, output only 50%, then list only the parts that contain init
P.sort_stats (' time ', ' cum '). Print_stats (. 5, ' init ')
#若想知道谁调用了上述函数可以使用
P.print_callers (. 5, ' init ')
*cprofile Module Description
Function:
Cprofile.run (Command, Filename=none, Sort=-1)
Cprofile.runctx (Command, Globals, locals, Filename=none)?
Class:
Cprofile.profile (Timer=none, timeunit=0.0, Subcalls=true, Builtins=true)
Methods under the Cprofile.profile class:
Enable ()
Start collecting profiling data.
Disable ()
Stop collecting profiling data.
Create_stats ()
Stop collecting profiling data and record the results internally as the current profile.
Print_stats (Sort=-1)
Create a Stats object based on the current profiles and print the results to stdout.
Dump_stats (filename)
Write the results of the current profiles to filename.
Run (CMD)
Profile the cmd via exec ().
Runctx (CMD, globals, locals)
The profile of the CMD via exec () with the specified global and local environment.
Runcall (func, *args, **kwargs)
Profile func (*args, **kwargs)
*stats Class (Pstats.stats) description
Strip_dirs () The path information before the file name is dropped.
Add (filename,[...]) Add the profile output file to the stats instance for statistics
Dump_stats (filename) Save the stats statistics to a file
Sort_stats (key,[...]) The most important function to sort the output of a profile
Reverse_order () Rearrange the data in the stats instance in reverse order
Print_stats ([restriction,...]) Export the stats report to stdout
Print_callers ([restriction,...]) Outputs information about the function that called the specified function
Print_callees ([restriction,...]) Outputs information about functions that have been called by the specified function
Sort_stats supports the following parameters:
' Calls ' Call Count
' Cumulative ' Cumulative time
' Cumtime ' Cumulative time
' File ' File name
' filename ' File name
' Module ' File name
' Ncalls ' Call Count
' Pcalls ' Primitive Call Count
' Line ' Line number
' Name ' Function name
' NFL ' Name/file/line
' Stdname ' Standard name
' Time ' Internal time
' Tottime ' Internal time
* A comparison of typical output results:
197 function calls (192 primitive calls) in 0.002 seconds
Ordered By:standard Name
Ncalls tottime percall cumtime percall filename:lineno (function)
1 0.000 0.000 0.001 0.001 <string>:1 (<module>)
1 0.000 0.000 0.001 0.001 re.py:212 (compile)
1 0.000 0.000 0.001 0.001 re.py:268 (_compile)
1 0.000 0.000 0.000 0.000 sre_compile.py:172 (_compile_charset)
1 0.000 0.000 0.000 0.000 sre_compile.py:201 (_optimize_charset)
4 0.000 0.000 0.000 0.000 sre_compile.py:25 (_identityfunction)
3/1 0.000 0.000 0.000 0.000 sre_compile.py:33 (_compile)
Output result Description:
There are 197 function calls, and the original call is 192 times, and the original invocation description does not contain a recursive call.
Sort by standard name. 3/1 indicates a recursive call has occurred, 1 is the original number of calls, and 3 is the number of recursive calls
The number of times the Ncalls function was called
Tottime function Total run time, removing function run time called in function
The average time that the Percall function runs once equals tottime/ncalls
Cumtime function Total run time, with call function run time
The average time that the Percall function runs once equals cumtime/ncalls
The name of the file in which the Filename:lineno function is located, the line number of the function, the name of the letter
Reference:
Https://docs.python.org/2/library/profile.html
Python Scripting Performance Profiling
