Memory leakage of < >python and analysis of GC module usage

Generally in Python, in order to solve the memory leak problem, the object reference count is used, and automatic garbage collection is implemented based on the reference count.
Because Python has automatic garbage collection function, it has caused a lot of beginners mistakenly think they have a good day, no longer be disturbed by memory leaks. But if you look closely at the description of the Python document to the __del__ () function, you know there is a cloud in this good day. Here's a little excerpt from the document below:

Some common situations that could prevent the reference count of an object from going to zero include:circular references B Etween objects (e.g., a doubly-linked list or a tree data structure with parent and child pointers); A reference to the object on the stack frame of a function this caught an exception (the traceback stored in Sys.exc_trace Back keeps the stack frame alive); Or a reference to the object on the stack frame, raised an unhandled exception in interactive mode (the Traceback stor Ed in sys.last_traceback keeps the stack frame alive).

As can be seen, circular references between objects with the __del__ () function are the main culprits that cause memory leaks.
It is also necessary to note that the circular reference between Python objects without the __del__ () function can be automatically garbage collected.

How do I know if an object is leaking memory?

Method One, when you think that an object should be destroyed (that is, the reference count is 0), you can use Sys.getrefcount (obj) to get the reference count of the object, and determine if the return value is zero for memory leaks. If the reference count returned is not 0, the object obj cannot be reclaimed by the garbage collector at this point.

Method Two, you can also use the Python Extension module GC to view the details of objects that cannot be reclaimed.

First, take a look at the normal test code:

#---------------code begin--------------#-*-coding:utf-8-*-ImportGCImportSYSclassCgcleak (object):def __init__(self): Self._text='#'*10def __del__(self):Passdefmake_circle_ref (): _gcleak=Cgcleak ()#_gcleak._self = _gcleak # test_code_1  Print '_gcleak ref count0:%d'%Sys.getrefcount (_gcleak)del_gcleakTry:    Print '_gcleak ref count1:%d'%Sys.getrefcount (_gcleak)exceptUnboundlocalerror:Print '_gcleak is invalid!'deftest_gcleak ():#Enable automatic garbage collection.gc.enable ()#Set The garbage collection debugging flags.Gc.set_debug (GC. debug_collectable | Gc. debug_uncollectable | /GC. Debug_instances|GC. debug_objects)Print 'begin leak Test ...'make_circle_ref ()Print 'begin collect ...'_unreachable=Gc.collect ()Print 'Unreachable Object num:%d'%_unreachablePrint 'Garbage Object num:%d'%Len (gc.garbage)if __name__=='__main__': Test_gcleak ()

In Test_gcleak (), after setting the garbage collector debug Flag, garbage collection is done with collect (), finally printing out the number of unreachable garbage objects found by the garbage collection and the number of garbage objects in the entire interpreter.

Gc.garbage is a list object, which is an object that the garbage collector finds unreachable (that is, garbage), but cannot be freed (that is, it cannot be reclaimed). The document is described as: A List of objects which the collector found to being unreachable but could is freed (Uncollectable objects).
In general, objects in Gc.garbage are objects in the reference ring. Because Python does not know what security order to invoke the __del__ () function of an object in the ring, the object always survives in gc.garbage, causing a memory leak. If you know a safe order, then break the reference ring and Execute del gc.garbage[:] To empty the list of junk objects.

The previous segment of the code output is (#后字符串为笔者所加注释):

#-----------------------------------------begin leak Test ...#the reference count for variable _gcleak is 2._gcleak ref Count0:2#_gcleak is an illegal variable that becomes unreachable (unreachable)._gcleak isinvalid!#Start garbage collectionbegin collect ...#The number of unreachable garbage objects found in this garbage collection is 0.Unreachable Object Num:0#the total number of garbage objects in the interpreter is 0.Garbage Object num:0#-----------------------------------------

This shows that the reference count of the _gcleak object is correct, and no memory leaks occur for any objects.

If you do not comment out the test_code_1 statement in Make_circle_ref ():

_gcleak._self = _gcleak

That is, let _gcleak form a circular reference to themselves. Run the above code again and the output will become:

# ----------------------------------------- begin leak Test..._gcleak ref COUNT0:3 is invalid!begin collect ... # found garbage objects that can be recycled: The address is 012aa090, and the type is cgcleak. Gc:uncollectable <cgcleak 012aa090><dict 012ac1e0>unreachable object num:2#  !! The number of garbage objects that cannot be reclaimed is 1, causing a memory leak! Garbage Object num:1#-----------------------------------------

Visible <cgcleak 012aa090> object has a memory leak!! And the more dict garbage is leaking _gcleak object dictionary, print out the dictionary information is:

{' _self ': <__main__. Cgcleak object at 0x012aa090>, ' _text ': ' ########## '}

In addition to a circular reference to itself, a circular reference between multiple objects can also cause a memory leak. A simple example is as follows:

#---------------code begin--------------classCgcleaka (object):def __init__(self): Self._text='#'*10def __del__(self):Passclasscgcleakb (object):def __init__(self): Self._text='*'*10def __del__(self):Passdefmake_circle_ref (): _a=Cgcleaka () _b=cgcleakb () _a._b= _b#test_code_2_b._a = _a#Test_code_3  Print 'ref count0:a=%d b=%d'% /(Sys.getrefcount (_a), Sys.getrefcount (_b))#_b._a = None # test_code_4  del_adel_bTry:    Print 'ref count1:a=%d'%Sys.getrefcount (_a)exceptUnboundlocalerror:Print '_a is invalid!'  Try:    Print 'ref count2:b=%d'%Sys.getrefcount (_b)exceptUnboundlocalerror:Print '_b is invalid!'#---------------code End----------------

This test results in the following output:

# ----------------------------------------- begin leak Test...ref count0:a=3 b=3are is <cgcleaka 012aa110 ><cgcleakb 012aa0b0><dict 012ac1e0><dict 012ac0c0>  Unreachable Object num:4Garbage Object num:2#------------------------------------ -----

A memory leak has occurred in the visible _a,_b object. Because the two are circular references, the garbage collector does not know how to recycle, that is, the __del__ () function that does not know which object to call first.

A memory leak can be avoided by breaking the ring reference using either of the following methods:

1. Comment out the test_code_2 statement in Make_circle_ref ();
2. Comment out the test_code_3 statement in Make_circle_ref ();
3. Uncomment the Test_code_4 statement in Make_circle_ref ().

The corresponding output turns into:

# ----------------------------------------- begin leak Test...ref count0:a # Note: The output here varies depending on the situation.  is   is invalid!begin collect...unreachable object Num:0garbage Object num:0#- ----------------------------------------

Conclusion: Python's GC has strong functions, such as setting Gc.set_debug (GC). Debug_leak) can be checked for memory leaks caused by circular references. If a memory leak check is made at development time, and when it is released to ensure that no memory leaks are available, you can increase the operational efficiency by extending the garbage collection interval for Python, or even proactively shutting down the garbage collection mechanism.

Memory leakage of < >python and analysis of GC module usage