Python performance splitting Tool

Python Performance splitting Tool

The project is about to be completed, but the tester told me that the performance test was not passed, which is not uncommon in development. The next task is to work overtime to find out the performance bottleneck, then optimize the performance, and then perform performance tests, so that the cycle continues until the performance test is passed. Although a wealth of work experience contributes to performance optimization, only scientific application tools can find the bottleneck of optimal optimization granularity in the shortest time.CodeSegment to get twice the result with half the effort.

Profile , Cprofile And Hotshot

PythonA variety of built-in performance optimization tools help us locate performance bottlenecks, such:Profile,CprofileAndHotshot. They are easy to use and have complete support documents for your reference. The most commonProfileThe module is used as an example to describe how to use them. Assume that the script file to be split isFoo. pyThe content is as follows:

Def Foo ():

Sum = 0

For I in range (100 ):

Sum + = I

Return sum

If _ name _ = "_ main __":

Foo ()

PairFoo. pyOne of the methods for splitting performance is to modifyFoo. pyInIfProgramBlock, introduceProfileModule:

If _ name _ = "_ main __":

Import Profile

Profile. Run ("Foo ()")

Then executeFoo. pyThe splitting result is printed as a text report to the standard output.

Because the above method needs to be modifiedFoo. pyFile, so we usually prefer to use the method without modifying the source file-that is, to use the application in the command line.PythonOf -MParameter to executeProfile:

Python-M profile Foo. py

In addition Profile You can also use Cprofile Module. Cprofile By C Language implementation is a tool with lower splitting cost. Profile The same module interface, but can only be used 2.5 Or later. Python Another built-in splitter is Hotshot , Hotshot The module is no longer recommended because it may be removed from the standard library in the future.

Pstats

No matter which splitter is used, its split data can be saved to a binary file, suchFoo. Prof. To analyze and view the splitting result file, usePstatsModule, which is highly scalable and can output a variety of text reports. It is an indispensable tool in the text interface.

UsePstatsThe analysis split result is very simple, just a few lines of code:

Import pstats

P = pstats. Stats ("foo. Prof ")

P. sort_stats ("time"). print_stats ()

Run the preceding script to generate a report that sorts the output result by the internal running time of the function (excluding the time when the sub-function is called.

Sort_stats ()The method isPstats. StatsOne of the most important methods is to sort the split data.Sort_stats ()Accept a string parameter, which identifies the sorting field. Common optional parameters and their meanings are as follows:

'ncall'	Number of calls
'cumulative '	total function running time
'nfl '	name/file/line
'time'	internal function running time (excluding the time when the subfunction is called)

BesidesSort_stats (),Pstats. StatsAndPrint_callees ()AndPrint_callers ()The method is used to output the functions called by the specified function and the functions that have called the specified function.

In addition to programming interfaces, Pstats It also provides a friendly command line interaction environment for command line execution Python-M pstats You can enter the interaction environment, and you can use read/ Add Command to read/load the splitting result file, Stats Command to view reports, Callees And Callers Command to view the callers and callers of a specific function. Yes Pstats , Identifies its basic usage: