Ways to manually release Python's memory

Source: Internet
Author: User
In the above optimizations, for every 500 users, some calculations are made and the results are recorded in the disk file. Originally thought to do so, these results are in the disk file, and will not continue to occupy memory, but in fact, Python's big pit is Python does not automatically clean up the memory. This is determined by its own realization. Specific reasons on-line multi-An article introduction, here will not copy.
This article will be affixed to a writer's experimental script, to illustrate that Python does not release the memory of the phenomenon, but also proposed a solution, namely: Del, and then explicitly call Gc.collect (). Script and specific effects are shown below.
Lab Environment One: Win 7, Python 2.7

From time import sleep, time import GC  def mem (way=1):  print Time () for  I in range (10000000):   if is = = = 1:    Pass   Else: # to 2, 3    del i      print time () if-to-  = 1 or the-to = = 2:   pass  Else: # to 3   g C.collect ()  print time ()    if __name__ = = "__main__":  print "Test" 1:just Pass "  mem (Way=1)  Sleep  print "test-2:just del"  Mem (way=2)  sleep  print "test-3:del, and then Gc.collec T () "  mem (way=3)  sleep (20)


The results of the operation are as follows:

Test 1:just pass 1426688589.47 1426688590.25 1426688590.25 test to 2:just del 1426688610.25 1426688611.05 142668861 1.05 Test 3:del, and then Gc.collect () 1426688631.05 1426688631.85 1426688631.95


For Way 1 and way 2, the result is exactly the same, the program memory consumption peak is 326772KB, in sleep 20 seconds, memory real-time consumption is 244820KB;
For the 3, program memory consumption peaks as above, but the real-time memory consumption of sleep is only 6336KB.
Lab Environment II: Ubuntu 14.10, Python 2.7.3
Operation Result:

Test 1:just pass 1426689577.46 1426689579.41 1426689579.41 test to 2:just del 1426689599.43 1426689601.1 1426689601 .1 Test-3:del, and then Gc.collect () 1426689621.12 1426689622.8 1426689623.11
ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py ubuntu 9134 0. 0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py ubuntu 9134 0. 0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 11.6 0.1 30956 5608 pts/1 s+ 14:39 0:05 python test_mem.py


Conclusion:
This shows that when del is called, Python does not actually release the memory, but instead keeps it in its memory pool, and only when the Gc.collect () is explicitly called, does the memory actually release.
Further:
In fact, go back to the previous blog script, also let it introduce Gc.collect (), and then write a monitoring script to monitor memory consumption situation:

while ((1)); Do Ps-aux | Sort-n-k5,6 | grep my_script; Free Sleep 5; Done


It turns out that memory does not recover after a group of 500 users, but continues to consume only about 70MB, and the GC appears to work. In this environment, the machine uses the cloud instance, the total memory 2G, the usable memory is approximately 1G, this script memory common consumption is 900m-1g. In other words, for this script, the GC does not work immediately, but the GC starts to function when the system's available memory drops from 1-1.2g to only about 70M. This is really strange, not knowing whether the script is related to the Gc.collect () used in thread, or that the GC is inherently not controllable. The author has not done the relevant experiments, may continue to discuss in the next blog post.
However, it is certain that if you do not use Gc.collect (), the original script will run out of system memory and be killed. This can be seen clearly in the syslog.

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.