Python calculates non-built-in data types that consume memory

Source: Internet
Author: User

The limitations of getsizeof

Python objects with non-built-in data types cannot get the true size with sys.getsizeof (), for example:

Import Networkx as NX Import  = for in xrange (10000)]print"size of L: " , Sys.getsizeof (L) g.add_nodes_from (L) Print " size of graph: ", sys.getsizeof (G)

Results

Size of l:8763264

Analysis

Graph graph contains point sequence l, and size is not as large as l, so it is not allowed to use getsizeof to calculate the size of Python's non-built-in type object.

Obtaining memory using the Psutil module

Example 1:

ImportNetworkx as NXImportPsutilImportSYSImportOsG=NX. Graph () L= [I forIinchXrange (10000)]Print "size of L:", Sys.getsizeof (L) g.add_nodes_from (L)Print "size of graph:", sys.getsizeof (G) process=Psutil. Process (Os.getpid ()) Max_mem=process.memory_info (). RSSPrint 'Max_mem:', Max_mem

There's a problem with this, and we need to get rid of the memory of the first system.

ImportPsutilImportSYSImportosprocess=Psutil. Process (Os.getpid ()) Max_mem_1=process.memory_info (). RSSPrint 'Max_mem:', max_mem_1g=NX. Graph () L= [I forIinchXrange (10000)]g.add_nodes_from (l) max_mem_2=process.memory_info (). RSSPrint 'max_2:', Max_mem_2Print 'Max_mem:', max_mem_2-max_mem_1

Results

max_mem:23724032316375047913472

Example 2:

ImportPsutilImportOSImportSYS fromDatetimeImportdatetimeprocess=Psutil. Process (Os.getpid ()) Max_mem_1= Process.memory_info (). rss/1024.0/1024.0/1024.0Print 'Max_mem 1:', Max_mem_1all_road_nx='a'* 1024 * 1024 * 1024 * 10;Print 'Size:', sys.getsizeof (ALL_ROAD_NX)/1024.0/1024.0/1024.0Print 'len All_road_nx:', Len (all_road_nx) max_mem_2= Process.memory_info (). rss/1024.0/1024.0/1024.0Print 'Max_mem 2:', Max_mem_2Print 'Max_mem 3:', max_mem_2-max_mem_1

Results:

Max_mem 1:0.0086250305175810.0000000345107374182402:10.00865173343: 10.0000267029
Python Module Psutil Introduction

The Psutil provides an interface that can be used to obtain information, including:

    • The currently running process
    • System (Resource usage) information
    • Cpu
    • Memory
    • Disk
    • Internet
    • User

Psutil implements a number of functions, including the following tools:

    • Ps
    • Top
    • Df
    • Kill
    • Free
    • Lsof
    • Free
    • Netstat
    • Ifconfig
    • Nice
    • Ionice
    • Iostat
    • Iotop
    • Uptime
    • Pidof
    • Tty
    • W.H.O.
    • Taskset
    • Pmap
Psutil Obtaining System Status examples
#! Coding:utf-8ImportNetworkx as NXImportPsutilImportSYSImportOSP=Psutil. Process (Os.getpid ()) Psutil.pids ()#View all system processesp = psutil. Process (6241)#View all system processesPrint "Name:", P.name ()#Process NamePrint "Bin Path", P.exe ()#the bin path of the processPrint "Process Absolute Path", P.CWD ()#The working directory absolute path of the processPrint "process Status", P.status ()#process StatusPrint "Process creation Time", P.create_time ()#Process creation TimePrint "Process UUID Information", P.uids ()#Process UID InformationPrint "Process GID Information", P.gids ()#GID information for the processPrint "CPU time information for the process", P.cpu_times ()#CPU time information for the process, including User,system two CPU informationPrint "get process CPU affinity", P.cpu_affinity ()#get process CPU affinity, if you want to set the CPU affinity, the CPU number as a reference is goodPrint "Process Memory Utilization", P.memory_percent ()#Process Memory UtilizationPrint "Process Memory Rss,vms information", P.memory_info ()#Process Memory Rss,vms informationPrint "IO information for the process", P.io_counters ()#IO information for the process, including read/write IO numbers and parametersPrint "Process List", P.connections ()#return to List of processesPrint "number of threads opened by the process", P.num_threads ()#number of threads opened by the process

Results

Name:pythonbin Path/home/tops/bin/python2.7Process Absolute Path/home/wzh94434/Iu_iso_test process state sleeping process creation time1463322002.74process UUID Information puids (real=124674, effective=124674, saved=124674) Process GID information pgids (real=100, effective=100, saved=100) Process CPU time information pcputimes (user=14.38, system=2.38, children_user=0.0, children_system=0.0) Get process CPU affinity [0,1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28] Process Memory Utilization0.0432284208934Process Memory Rss,vms information Pmem (RSS=58548224, vms=534482944, shared=6922240, text=1536000, lib=0, data=268894208, dirty=0) Process IO information pio (Read_count=4166, write_count=1192, Read_bytes=0, write_bytes=0) Process list [Pconn (FD=3, family=2, type=1, Laddr= ('10.184.70.11', 57785), raddr= ('10.184.70.13', 8018), status='established')] The number of threads that the process is opening4

Python calculates non-built-in data types that consume memory

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.