Talk about how to manually free python memory _python

In the previous blog, you mentioned multiple optimizations for a script. I thought it had been optimized, but when the tester tested it, I realized that I had stepped into a big hole in Python.

In the optimization above, for every 500 users, some calculations are performed and the results are recorded in the disk file. Originally thought to do so, these results in the disk file, and will not continue to occupy memory, but in fact, Python's big pit is Python does not automatically clean up the memory. This is determined by its own realization. Specific reasons on the Internet more an article introduction, here is not copy.

This blog will post a writer's experimental script to illustrate that Python does not release the memory of the phenomenon, but also proposed a solution, namely: First Del, and then explicitly call Gc.collect (). Script and concrete effects see below.

Experimental environment One: Win 7, Python 2.7

From time import sleep, time 
import GC 
 
def mem (way=1): 
 print Time () to 
 I in range (10000000): 
  If way = = 1: 
   pass 
  Else: # way 2, 3 
   del i 
    
 print time () 
 if way = 1 or way = 2: 
  pass 
 else: # way 3< C14/>gc.collect () 
 print time () 
   
if __name__ = = "__main__": 
 print "Test way 1:just Pass" 
 Mem (Way=1) C19/>sleep 
 print "Test way 2:just del" 
 mem (way=2) Sleep 
 () 
 print "Test way 3:del, and then GC.C Ollect () " 
 mem (way=3) sleep 
 (20) 
  

The results of the operation are as follows:

Test way 1:just Pass 
1426688589.47 
1426688590.25 
1426688590.25 
Test way 2:just del 
1426688610.25 
1426688611.05 
1426688611.05 
Test Way 3:del, and then Gc.collect () 
1426688631.05 
1426688631.85 

For Way 1 and way 2, the result is exactly the same, the program memory consumption peak is 326772KB, in sleep 20 seconds, memory real-time consumption is 244820KB;

For Way 3, the peak memory consumption of the program is the same, but the real-time memory consumption of sleep is only 6336KB.

Experimental Environment II: Ubuntu 14.10, Python 2.7.3

Run Result:

Test way 1:just Pass 
1426689577.46 
1426689579.41 
1426689579.41 
Test way 2:just del 
1426689599.43 
1426689601.1 
1426689601.1 
Test Way 3:del, and then Gc.collect () 
1426689621.12 
1426689622.8 

ubuntu@my_machine:~$ Ps-aux | grep test_mem 
Warning:bad PS syntax, perhaps a bogus '-'?  http://procps.sf.net/faq.html 
Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py 
ubuntu 9134 0.0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ ps-aux 
| grep test_mem
   
    warning:bad PS syntax, perhaps a bogus '-'?  http://procps.sf.net/faq.html 
Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py 
ubuntu 9134 0.0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ ps-aux 
| grep test_mem
    
     warning:bad PS syntax, perhaps a bogus '-'? Http://procps.sf.net/faq.html 
    
   

Conclusion:

As it turns out, when you call Del, Python does not actually release the memory, but instead continues to put it in its memory pool, and only when the Gc.collect () is explicitly invoked will the memory be really released.

Further:

In fact, go back to the previous blog script, also let it introduce Gc.collect (), and then write a monitoring script to monitor memory consumption:

The result is that the GC does not appear to be working until the memory is recovered in a group of 500 users, but it continues to consume only about 70MB. In this environment, the machine uses the cloud instance, the total memory 2G, the usable memory is approximately 1G, this script memory common consumption is 900m-1g. In other words, for this script, the GC does not work immediately, but only when the system's available memory drops from 1-1.2g down to around 70M, the GC does not begin to function. This is really odd, and it's not clear whether the script is related to the Gc.collect () used in thread, or that the GC works is inherently not controllable. The author has not done related experiments, may continue to explore in the next blog.

However, it is certain that if you do not use Gc.collect (), the original script will run out of system memory and be killed. This can be seen from the syslog.

The above is the entire content of this article, I hope to help you learn, but also hope that we support the cloud habitat community.