10 ways to detect Python program run time, CPU, and memory consumption

When you run a complex Python program, the execution time is long, and you might want to improve the execution efficiency of your program. But what should we do?

First, there is a tool that detects bottlenecks in your code, for example, to find out which part of the execution time is longer. Next, it is optimized for this part.

At the same time, you need to control memory and CPU usage so that you can optimize your code on the other.

Therefore, in this article I will introduce 7 different Python tools to check the execution time of functions in the code and the use of memory and CPU.
1. Use adorners to measure function execution time

There is an easy way to define an adorner to measure the execution time of a function and output the result:

Import timefrom functools Import wraps def fn_timer (function):  @wraps (function)  def function_timer (*args, * * Kwargs):    t0 = time.time ()    result = function (*args, **kwargs)    t1 = time.time ()    print ("Total time running %s:%s seconds "%        (Function.func_name, str (t1-t0))        )    return result  return Function_timer

Next, add the adorner to the function you want to measure, as follows:

@fn_timerdef myfunction (...): ...

For example, here is a function that detects the time required to sort an array containing 2 million random numbers:

@fn_timerdef random_sort (n):  return sorted ([Random.random () for I in range (n)]) if __name__ = = "__main__":  Random_sort (2000000)

When you execute the script, you see the following result:

Total time running random_sort:1.41124916077 seconds

2. Using the Timeit module

Another approach is to use the Timeit module, which calculates the average time consumption.

Execute the following script to run the module.

Python-m timeit-n 4-r 5-s "Import timing_functions" "Timing_functions.random_sort (2000000)"

The timing_functions here is the Python script file name.

At the end of the output, you can see the following results:

4 loops, Best of 5:2.08 sec per loop

This means that the test has been tested 4 times, the average test is repeated 5 times, the best test result is 2.08 seconds.

If you do not specify a test or number of repetitions, the default is 10 Tests, repeating 5 times at a time.
3. Use the time command from the UNIX system

However, adorners and Timeit are all based on Python. The Unix time utility is useful when testing python in an external environment.

Run the time utility:

$ time-p Python timing_functions.py

The output is:

Total time running random_sort:1.3931210041 secondsreal 1.49user 1.40sys 0.08

The first line comes from the predefined adorner, and the other three behaviors:

• Real represents the total time the script was executed
• User represents the CPU time consumed by the execution script.
• SYS represents the time spent executing kernel functions.

Note: According to Wikipedia, the kernel is a computer program that manages the input and output of the software and translates it into data processing instructions that can be executed by the electronic devices in the CPU and other computers.

Therefore, the difference between real execution time and User+sys execution time is the time consumed by input/output and other tasks of the system.
4. Using the Cprofile module

If you want to know how much time each function and method consumes, and how many times these functions are called, you can use the Cprofile module.

$ python-m cprofile-s Cumulative timing_functions.py

You can now see a detailed description of the functions in your code that contain the number of times each function is called, because the-s option (additive) is used, and the final result is sorted according to the cumulative execution time of each function.

The reader will find that the total time required to execute the script is more than before. This is due to the fact that it takes time to measure the execution time of each function.
5. Using the Line_profiler module

The Line_profiler module gives you the CPU time required to execute each line of code.

First, install the module:

$ pip Install Line_profiler

Next, you need to specify which function to detect with @profile (you do not need to import the module in your code):

@profiledef random_sort2 (N):  l = [Random.random () for I in range (n)]  l.sort ()  return L if __name__ = = "__main_ _ ":  random_sort2 (2000000)

It is best to obtain a line-by-row description of the RANDOM_SORT2 function with the following command.

$ kernprof-l-V timing_functions.py

Where-L is interpreted as a row-by-line explanation, and-V indicates output verbose results. In this way, we see that the build array consumes 44% of the computation time, while the sort () method consumes the remaining 56% of the time.

Similarly, the execution time of the script takes longer because of the need to detect execution time.
6. Using the Memory_profiler module

The Memory_profiler module is used for memory usage based on progressive measurement code. Using this module will make the code run more slowly.

The installation method is as follows:

Pip Install Memory_profiler

Also, it is recommended that you install the Psutil package so that the Memory_profile will run faster:

$ pip Install Psutil

Similar to Line_profiler, use the @profile adorner to identify functions that need to be traced. Next, enter:

$ python-m Memory_profiler timing_functions.py

The execution time of the script is 1 or 2 seconds longer than before. If the Psutil package is not installed, it may be longer.

As can be seen from the results, memory usage is measured in MIB, represented by mebibyte (1MiB = 1.05MB).
7. Using the Guppy Package

Finally, this package lets you know how many objects each type (str, tuple, dict, and so on) has created in each phase of code execution.

The installation method is as follows:

$ pip Install Guppy

Next, add it to your code:

From Guppy import hpy def random_sort3 (n):  hp = hpy ()  print "Heap at the beginning of the Functionn", Hp.heap () 
  l = [Random.random () for I in range (n)]  l.sort ()  print "Heap at the end of the Functionn", Hp.heap ()  return L if __name__ = = "__main__":  random_sort3 (2000000)

To run the code:

$ python timing_functions.py

You can see the output as:

By placing the heap () in a different location in your code, you can learn about the flow of object creation and deletion operations in the script.

If you want to learn more about Python code speed optimization, I suggest you read this book "High performance python:practical performant programming for humans, September 2014. "

I hope this article can help you! ^_^

