In the above optimizations, for every 500 users, some calculations are made and the results are recorded in the disk file. Originally thought to do so, these results are in the disk file, and will not continue to occupy memory, but in fact, Python's big pit is Python does not automatically clean up the memory. This is determined by its own realization. Specific reasons on-line multi-An article introduction, here will not copy.
This article will be affixed to a writer's experimental script, to illustrate that Python does not release the memory of the phenomenon, but also proposed a solution, namely: Del, and then explicitly call Gc.collect (). Script and specific effects are shown below.
Lab Environment One: Win 7, Python 2.7
From time import sleep, time import GC def mem (way=1): print Time () for I in range (10000000): if is = = = 1: Pass Else: # to 2, 3 del i print time () if-to- = 1 or the-to = = 2: pass Else: # to 3 g C.collect () print time () if __name__ = = "__main__": print "Test" 1:just Pass " mem (Way=1) Sleep print "test-2:just del" Mem (way=2) sleep print "test-3:del, and then Gc.collec T () " mem (way=3) sleep (20)
The results of the operation are as follows:
Test 1:just pass 1426688589.47 1426688590.25 1426688590.25 test to 2:just del 1426688610.25 1426688611.05 142668861 1.05 Test 3:del, and then Gc.collect () 1426688631.05 1426688631.85 1426688631.95
For Way 1 and way 2, the result is exactly the same, the program memory consumption peak is 326772KB, in sleep 20 seconds, memory real-time consumption is 244820KB;
For the 3, program memory consumption peaks as above, but the real-time memory consumption of sleep is only 6336KB.
Lab Environment II: Ubuntu 14.10, Python 2.7.3
Operation Result:
Test 1:just pass 1426689577.46 1426689579.41 1426689579.41 test to 2:just del 1426689599.43 1426689601.1 1426689601 .1 Test-3:del, and then Gc.collect () 1426689621.12 1426689622.8 1426689623.11
ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py ubuntu 9134 0. 0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 10.0 6.0 270916 245564 pts/1 s+ 14:39 0:03 python test_mem.py ubuntu 9134 0. 0 0.0 8104 924 pts/2 s+ 14:40 0:00 grep--color=auto test_mem ubuntu@my_machine:~$ Ps-aux | grep test_mem Warning:bad PS syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html Ubuntu 9122 11.6 0.1 30956 5608 pts/1 s+ 14:39 0:05 python test_mem.py
Conclusion:
This shows that when del is called, Python does not actually release the memory, but instead keeps it in its memory pool, and only when the Gc.collect () is explicitly called, does the memory actually release.
Further:
In fact, go back to the previous blog script, also let it introduce Gc.collect (), and then write a monitoring script to monitor memory consumption situation:
while ((1)); Do Ps-aux | Sort-n-k5,6 | grep my_script; Free Sleep 5; Done
It turns out that memory does not recover after a group of 500 users, but continues to consume only about 70MB, and the GC appears to work. In this environment, the machine uses the cloud instance, the total memory 2G, the usable memory is approximately 1G, this script memory common consumption is 900m-1g. In other words, for this script, the GC does not work immediately, but the GC starts to function when the system's available memory drops from 1-1.2g to only about 70M. This is really strange, not knowing whether the script is related to the Gc.collect () used in thread, or that the GC is inherently not controllable. The author has not done the relevant experiments, may continue to discuss in the next blog post.
However, it is certain that if you do not use Gc.collect (), the original script will run out of system memory and be killed. This can be seen clearly in the syslog.