Introduction and usage of two Python memory detection tools

This article mainly introduces the introduction and usage of two Python memory detection tools, which can be used to analyze the memory usage of Python programs. If you need a friend, refer to when I wrote a program last year, if you are not sure about your memory usage, you can find a write tool to print the memory usage of the program or function.
Here, we will record the basic usage of the two memory detection tools we found last time. It is also necessary to analyze the memory usage of the Python program in the future.

Memory_profiler module (used with psutil)
Note: I like psutil, which implements the main functions of many Linux commands, such as ps, top, lsof, netstat, ifconfig, who, df, kill, free.
Sample Code (https://github.com/smilejay/python/blob/master/py2014/mem_profile.py ):

The Code is as follows:

#! /Usr/bin/env python

'''
Created on May 31,201 4

@ Author: Jay
@ Description: use memory_profiler module for profiling programs/functions.
'''

From memory_profiler import profile
From memory_profiler import memory_usage
Import time


@ Profile
Def my_func ():
A = [1] * (10 ** 6)
B = [2] * (2*10 ** 7)
Del B
Return


Def cur_python_mem ():
Mem_usage = memory_usage (-1, interval = 0.2, timeout = 1)
Return mem_usage


Def f (a, n = 100 ):
Time. sleep (1)
B = [a] * n
Time. sleep (1)
Return B

If _ name _ = '_ main __':
A = my_func ()
Print cur_python_mem ()
Print ""
Print memory_usage (f, (1,), {'N': int (1e6)}), interval = 0.5)

Run the above Code and the output result is:

The Code is as follows:

Jay @ Jay-Air :~ /Workspace/python. git/py2014 $ python mem_profile.py
Filename: mem_profile.py

Line # Mem usage Increment Line Contents
========================================================== ==========
15 8.0 MiB 0.0 MiB @ profile
16 def my_func ():
17 15.6 MiB 7.6 MiB a = [1] * (10 ** 6)
18 168.2 MiB 152.6 MiB B = [2] * (2*10 ** 7)
19 15.6 MiB-152.6 MiB del B
20 15.6 MiB 0.0 MiB return


[15.61328125, 15.6171875, 15.6171875, 15.6171875, 15.6171875]

[15.97265625, 16.00390625, 16.00390625, 17.0546875, 23.63671875, 23.63671875, 23.640625]

Guppy (Heapy used)
Guppy is an umbrella package combining Heapy and GSL with support utilities such as the Glue module that keeps things together.
Sample Code (https://github.com/smilejay/python/blob/master/py2014/try_guppy.py ):

The Code is as follows:

#! /Usr/bin/env python

'''
Created on May 31,201 4

@ Author: Jay

@ Description: just try to use Guppy-PE (useing Heapy) for memory profiling.
'''


From guppy import hpy

A = [8] * (10 ** 6)

H = hpy ()
Print h. heap ()
Print h. heap (). more
Print h. heap (). more. more

Note that more information is output in. more usage.
Run the above program and the output result is:

The Code is as follows:

Jay @ Jay-Air :~ /Workspace/python. git/py2014 $ python try_guppy.py
Partition of a set of 26963 objects. Total size = 11557848 bytes.
Index Count % Size % Cumulative % Kind (class/dict of class)
0 177 1 8151560 71 8151560 71 list
1 12056 45 996840 9 9148400 79 str
2 5999 22 488232 4 9636632 83 tuple
3 324 1 283104 2 9919736 86 dict (no owner)
4 68 0 216416 2 10136152 88 dict of module
5 199 1 210856 2 10347008 90 dict of type
6 1646 6 210688 2 10557696 91 types. CodeType
7 1610 6 193200 2 10750896 93 function
8 199 1 177008 2 10927904 95 type
9 124 0 135328 1 11063232 96 dict of class
<91 more rows. Type e.g. '_. more' to view.>
Index Count % Size % Cumulative % Kind (class/dict of class)
10 1045 4 83600 1 11148456 96 _ builtin _. wrapper_descriptor
11 109 0 69688 1 11218144 97 dict of guppy. etc. Glue. Interface
12 389 1 34232 0 11252376 97 _ builtin _. weakref
13 427 2 30744 0 11283120 97 types. BuiltinFunctionType
14 411 2 29592 0 11312712 98 _ builtin _. method_descriptor
15 25 0 26200 0 11338912 98 dict of guppy. etc. Glue. Share
16 108 0 25056 0 11363968 98 _ builtin _. set
17 818 3 19632 0 11383600 98 int
18 66 0 18480 0 11402080 98 dict of guppy. etc. Glue. Owner
19 16 0 17536 0 11419616 99 dict of abc. ABCMeta
<81 more rows. Type e.g. '_. more' to view.>
(Some output is omitted later)

In addition, there is also a memory profiling called "PySizer", but it is not maintained.