Introduction and usage of two Python memory detection tools

Source: Internet
Author: User
Tags builtin
This article mainly introduces the introduction and usage of two Python memory detection tools, which can be used to analyze the memory usage of Python programs. If you need a friend, refer to when I wrote a program last year, if you are not sure about your memory usage, you can find a write tool to print the memory usage of the program or function.
Here, we will record the basic usage of the two memory detection tools we found last time. It is also necessary to analyze the memory usage of the Python program in the future.

Memory_profiler module (used with psutil)
Note: I like psutil, which implements the main functions of many Linux commands, such as ps, top, lsof, netstat, ifconfig, who, df, kill, free.
Sample Code (https://github.com/smilejay/python/blob/master/py2014/mem_profile.py ):

The Code is as follows:


#! /Usr/bin/env python

'''
Created on May 31,201 4

@ Author: Jay
@ Description: use memory_profiler module for profiling programs/functions.
'''

From memory_profiler import profile
From memory_profiler import memory_usage
Import time


@ Profile
Def my_func ():
A = [1] * (10 ** 6)
B = [2] * (2*10 ** 7)
Del B
Return


Def cur_python_mem ():
Mem_usage = memory_usage (-1, interval = 0.2, timeout = 1)
Return mem_usage


Def f (a, n = 100 ):
Time. sleep (1)
B = [a] * n
Time. sleep (1)
Return B

If _ name _ = '_ main __':
A = my_func ()
Print cur_python_mem ()
Print ""
Print memory_usage (f, (1,), {'N': int (1e6)}), interval = 0.5)

Run the above Code and the output result is:

The Code is as follows:


Jay @ Jay-Air :~ /Workspace/python. git/py2014 $ python mem_profile.py
Filename: mem_profile.py

Line # Mem usage Increment Line Contents
========================================================== ==========
15 8.0 MiB 0.0 MiB @ profile
16 def my_func ():
17 15.6 MiB 7.6 MiB a = [1] * (10 ** 6)
18 168.2 MiB 152.6 MiB B = [2] * (2*10 ** 7)
19 15.6 MiB-152.6 MiB del B
20 15.6 MiB 0.0 MiB return


[15.61328125, 15.6171875, 15.6171875, 15.6171875, 15.6171875]

[15.97265625, 16.00390625, 16.00390625, 17.0546875, 23.63671875, 23.63671875, 23.640625]

Guppy (Heapy used)
Guppy is an umbrella package combining Heapy and GSL with support utilities such as the Glue module that keeps things together.
Sample Code (https://github.com/smilejay/python/blob/master/py2014/try_guppy.py ):

The Code is as follows:


#! /Usr/bin/env python

'''
Created on May 31,201 4

@ Author: Jay

@ Description: just try to use Guppy-PE (useing Heapy) for memory profiling.
'''


From guppy import hpy

A = [8] * (10 ** 6)

H = hpy ()
Print h. heap ()
Print h. heap (). more
Print h. heap (). more. more

Note that more information is output in. more usage.
Run the above program and the output result is:

The Code is as follows:


Jay @ Jay-Air :~ /Workspace/python. git/py2014 $ python try_guppy.py
Partition of a set of 26963 objects. Total size = 11557848 bytes.
Index Count % Size % Cumulative % Kind (class/dict of class)
0 177 1 8151560 71 8151560 71 list
1 12056 45 996840 9 9148400 79 str
2 5999 22 488232 4 9636632 83 tuple
3 324 1 283104 2 9919736 86 dict (no owner)
4 68 0 216416 2 10136152 88 dict of module
5 199 1 210856 2 10347008 90 dict of type
6 1646 6 210688 2 10557696 91 types. CodeType
7 1610 6 193200 2 10750896 93 function
8 199 1 177008 2 10927904 95 type
9 124 0 135328 1 11063232 96 dict of class
<91 more rows. Type e.g. '_. more' to view.>
Index Count % Size % Cumulative % Kind (class/dict of class)
10 1045 4 83600 1 11148456 96 _ builtin _. wrapper_descriptor
11 109 0 69688 1 11218144 97 dict of guppy. etc. Glue. Interface
12 389 1 34232 0 11252376 97 _ builtin _. weakref
13 427 2 30744 0 11283120 97 types. BuiltinFunctionType
14 411 2 29592 0 11312712 98 _ builtin _. method_descriptor
15 25 0 26200 0 11338912 98 dict of guppy. etc. Glue. Share
16 108 0 25056 0 11363968 98 _ builtin _. set
17 818 3 19632 0 11383600 98 int
18 66 0 18480 0 11402080 98 dict of guppy. etc. Glue. Owner
19 16 0 17536 0 11419616 99 dict of abc. ABCMeta
<81 more rows. Type e.g. '_. more' to view.>
(Some output is omitted later)

In addition, there is also a memory profiling called "PySizer", but it is not maintained.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.