How to manually release Python memory

In the above optimization, some calculations are performed for every 500 users and the results are recorded in the disk file. These results are stored in the disk file instead of memory usage. but in fact, Python does not automatically clean up the memory. This is determined by its own implementation. For specific reasons, there are many articles on the Internet, so we will not copy them here. In the above optimization, some calculations are performed for every 500 users and the results are recorded in the disk file. These results are stored in the disk file instead of memory usage. but in fact, Python does not automatically clean up the memory. This is determined by its own implementation. For specific reasons, there are many articles on the Internet, so we will not copy them here.
This article will post an experiment script by the author to demonstrate that Python does not release memory, and also proposes a solution, namely, del first and then explicitly call gc. collect (). for the script and the specific effect, see.
Lab environment 1: Win 7, Python 2.7

from time import sleep, time import gc  def mem(way=1):  print time()  for i in range(10000000):   if way == 1:    pass   else: # way 2, 3    del i      print time()  if way == 1 or way == 2:   pass  else: # way 3   gc.collect()  print time()    if __name__ == "__main__":  print "Test way 1: just pass"  mem(way=1)  sleep(20)  print "Test way 2: just del"  mem(way=2)  sleep(20)  print "Test way 3: del, and then gc.collect()"  mem(way=3)  sleep(20)

The running result is as follows:

Test way 1: just pass 1426688589.47 1426688590.25 1426688590.25 Test way 2: just del 1426688610.25 1426688611.05 1426688611.05 Test way 3: del, and then gc.collect() 1426688631.05 1426688631.85 1426688631.95

For way 1 and way 2, the results are exactly the same. The program's memory consumption peak is 326772KB. when sleep is 20 seconds, the real-time memory consumption is 244820KB;
For way 3, the program memory consumption peak is the same as above, but the real-time memory consumption during sleep is only KB.
Lab environment 2: Ubuntu 14.10, Python 2.7.3
Running result:

Test way 1: just pass 1426689577.46 1426689579.41 1426689579.41 Test way 2: just del 1426689599.43 1426689601.1 1426689601.1 Test way 3: del, and then gc.collect() 1426689621.12 1426689622.8 1426689623.11

ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 10.0 6.0 270916 245564 pts/1 S+ 14:39 0:03 python test_mem.py ubuntu 9134 0.0 0.0 8104 924 pts/2 S+ 14:40 0:00 grep --color=auto test_mem ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 10.0 6.0 270916 245564 pts/1 S+ 14:39 0:03 python test_mem.py ubuntu 9134 0.0 0.0 8104 924 pts/2 S+ 14:40 0:00 grep --color=auto test_mem ubuntu@my_machine:~$ ps -aux | grep test_mem Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html ubuntu 9122 11.6 0.1 30956 5608 pts/1 S+ 14:39 0:05 python test_mem.py

Conclusion:
As described above, when del is called, Python does not actually release the memory, but instead stores it in its memory pool. only explicitly calls gc. when collect () is used, the real release memory is used.
Further:
In fact, back to the script of the previous blog, also let it introduce gc. collect (), and then write a monitoring script to monitor memory consumption:

while ((1)); do ps -aux | sort -n -k5,6 | grep my_script; free; sleep 5; done

The results show that the memory is not restored after the execution of a group of 500 users, but is continuously consumed until the remaining 70 MB, gc seems to work. In this environment, the machine uses a Cloud instance with a total memory of 2 GB and a usable memory of about 1 GB. the memory commonly consumed by this script is 900 MB-1 GB. In other words, gc does not take effect immediately for this script. Instead, gc takes effect only when the available system memory drops from 1-GB to about 70 MB. This is indeed strange. I don't know whether the script is related to gc. collect () used in the Thread, or whether gc is originally not controllable. I have not conducted any relevant experiments, and may continue to explore them in the next blog.
However, if gc. collect () is not used, the original script will consume the system memory and be killed. This is obvious from syslog.

The above describes how to manually release Python memory. For more information, see other related articles in the first PHP community!