Python script analyzes CPU usage

In this article, I'll discuss a tool for analyzing CPU usage in Python. CPU analysis measures the performance of the code by analyzing how the CPU executes the code to find the irregularities in the code and then process them.

Next we'll look at how to track the CPU usage of the Python script, focusing on the following areas:

1, CProfile

2, Line_profiler

3, Pprofile

4, Vprof

Measuring CPU Usage

For this article, I will use the same script as the one used in memory analysis, as follows:

Also, keep in mind that in PyPy2, you need to use the PIP version with which you want to match:

and other dependencies will also be installed:

CProfile

One of the most commonly used tools when discussing CPU analysis is cprofile, mainly because it is built into CPython2 and PyPy2. This is a deterministic parser, which means that a set of statistics is collected when the program is run, such as the number of executions or execution times for each part of our code. Additionally, Cprofile is less expensive on the system than other built-in analyzers (configuration files).

The use of CPython2 is simple:

If you use PyPy2:

The output is as follows:

Even with this text output, it is easy to see that our script has called the List.append method multiple times.

If we use Gprof2dot, we can see the cprofile output graphically. To use it, we must first install Graphviz, then some dependent packages, and finally use the following command on Ubuntu:

Run the script again:

We get the following output.png files:

This makes it easier to see everything. Let's take a closer look at its output. You can see that the function calls in the script are as follows:

1. First line: Python file name, line number, and method name

2. Second line: The percentage of code blocks that occupy all time

3, the third line: in parentheses, the method itself as a percentage of the total time

4. Line fourth: Number of times the function was called

For example, in the third Red Square at the top, the method primes takes 98.28% of the time, with 65.44% of the content in it, and the call 40 times. The rest of the time is spent in Python's List.append (22.33%) and Range (11.51%).

As a simple script, we just need to rewrite our script as follows:

If we use CPython2 to measure the time of our script,

There are also PyPy2:

We got good results by using PyPy2 's CPython2 and 3.1X, and here's the call flowchart for Cprofile:

You can also use cprofile programmatically, for example:

This is useful in some cases, such as multi-process performance measurements

Line_profiler

This parser provides information about the workload at the row level. It is implemented using Cython in C and compares it to Cprofile to find it has a small overhead.

The source code can be found here, or you can find the PyPI page here. Compared to Cprofile, it has the same overhead, but it takes 12 times times the time to get the configuration file.

To use it, you need to add it via PIP First: Pip install pip install cython Ipython = = 5.4.1 Line_profiler (CPython2). One of the main drawbacks of this parser is that it does not support pypy.

Just like with Memory_profiler, you need to add an adorner to the function you want to analyze. In our example, you need to add @profile before defining our primes function in 03.primes-v1.py. Then call it:

You will get the following output:

We see that repeated calls to List.append's two loops took the most time.

Pprofile

According to the author, Pprofile is a "pure Python parser for thread measurement and statistics".

It is inspired by Line_profiler and fixes a number of drawbacks, but it can also be used successfully with PyPy because it is written entirely in Python. Compared with cprofile, the analysis time of using CPython is 28 times times more, while using pypy, the analysis time is 10 times times more, and the detail level is more detailed.

We also support pypy! In addition, it supports parsing threads, which can be handy in a variety of situations.

To use it, you need to add it via PIP First: Pip install Pprofile (CPython2)/pypy-m pip install Pprofile (pypy), and then call it:

The output differs from what we saw before, and we get the following result:

We can now see everything in more detail. Let's take a look at the output. You can get the entire output of the script, and before each line you can see the number of calls made to it, the elapsed time (in seconds), the time of each call and the percentage of global time, and Pprofile adds additional lines to our output (such as lines 44th and 50, starting with (call) and accumulating metrics.

Again, we see that repeated calls to List.append's two loops cost the most time in our script.

Vprof

Vprof is a Python parser that provides rich interactive visualizations for a variety of Python program features, such as runtime and memory usage. It is a graphical display of node. JS based on the results displayed in the Web page.

With it, you can see one or all of the following related to the Python script:

1. CPU Usage diagram

2. Code Analysis

3. Memory diagram

4. Code Heat Map

To use it, you need to add it via PIP First: Pip install Vprof (CPython2)/pypy-m pip install Vprof (pypy), and then call it:

On CPython2, display the Code heat map (the first call follows) and code analysis (the second call below):

On PyPy, display the Code heat map (the first call follows) and code analysis (the second call below):

In each case, you'll see the following in the code scatter plot

And the following for code analysis.

The results are graphically visible, and we can hover over the mouse or click each line for more information. Again, we see that repeated calls to List.append's two loops cost the most time in our script.

English Original: https://pythonfiles.wordpress.com/2017/06/01/hunting-performance-in-python-code-part-3/

Translator: Buhaoxuesheng

Python script analyzes CPU usage