Python garbage collection mechanism

First, the Python garbage collection mechanism

Garbage collection in Python is based on reference counts and is supplemented by generational collections.

In Python, if an object has a reference count of 0,python, the memory of the object is reclaimed by the virtual machine.

classA:def __init__(self): self.t=NonePrint 'new obj, ID is%s'%Str (hex ( self))def __del__(self):Print 'del obj, id ID%s'%Str (hex ( self))if __name__=='__main__':     whileTRUE:A1=A ()delA1

Running the code above, the memory that the process consumes is basically not changing

 is 0x2a79d48ldel obj, ID ID 0x2a79d48l

A = A () creates an object in 0x2a79d48l memory where a variable points to this memory, when the reference count for this memory is 1

After del A, the a variable no longer points to 0x2a79d48l memory, so the reference count of this block of memory is reduced by one, equal to 0, so the object is destroyed and the memory is freed.

Case that causes reference count +1

1) objects are created, such as a=23

2) objects are referenced, such as B=a

3) The object is passed into a function as a parameter, such as Func (a)

4) object as an element, stored in a container, e.g. List1=[a,a]

Case that causes reference count-1

1) An alias for an object is explicitly destroyed, for example, Del a

2) The alias of the object is assigned to a new object, such as a=24

3) When an object leaves its scope, such as when the F function finishes executing, the local variable in the Func function (global variable does not)

4) The container in which the object is located is destroyed, or the object is deleted from the container

Second, demo view reference count

deffunc (c):Print 'In func function', Sys.getrefcount (c)-1Print 'Init', Sys.getrefcount (11)-1a= 11Print 'After a=11', Sys.getrefcount (11)-1b=aPrint 'After b=a', Sys.getrefcount (11)-1func (11)Print 'After func (a)', Sys.getrefcount (11)-1List1= [A, 12, 14]Print 'After list1=[a,12,14]', Sys.getrefcount (11)-1a=12Print 'After a=12', Sys.getrefcount (11)-1delaPrint 'After del a', Sys.getrefcount (11)-1delbPrint 'After del b', Sys.getrefcount (11)-1list1.pop (0)Print 'After pop list1', Sys.getrefcount (11)-1delList1Print 'After del List1', Sys.getrefcount (11)-1

Output:

Init, aftera=11, afterb=a, in func function, afterlist1=[a,12,14] Aftera=12del A,del b.  List1 67

Question: Why does the calling function make reference count +2

Sys.getrefcount (a) can view the reference count of a object, but it is 1 larger than the normal count, because a is passed in when the function is called, which gives a reference count of +1

Third, circular references cause memory leaks

classA:def __init__(self): self.t=NonePrint 'new obj, ID is%s'%Str (hex ( self))def __del__(self):Print 'del obj, id ID%s'%Str (hex ( self))if __name__=='__main__':     whileTRUE:A1=A () A2=A () a1.t=A2 a2.t=A1delA1delA2

The memory used by the process increases as the code is executed.

 is 0x3475348l are 0x3475388l

After creating the A1,A2, 0x3475348l (a1 corresponding memory, Memory 1), 0x3475388l (c2 corresponding memory, memory 2) both memory reference count is 1, execute a1.t = a2 and a2.t = A1, the reference count of the two pieces of memory becomes 2.

After del A1, the reference count of the memory 1 object becomes 1, because it is not 0, so the memory 1 object is not destroyed, so the number of references to memory 2 is still 2, after Del C2, in the same vein, memory 1 object, memory 2 object reference count is 1.

Although all two of their objects can be destroyed, the garbage collector does not recycle them because of the circular references, which results in a memory leak.

Of course, this is a special case, because ClassA defines the __del__ (self): function, and there is a circular reference, which has a circular reference to this case, the GC is not sure to call that Del First, so it cannot be recycled. If ClassA does not define the __del__ (self): function, the GC is able to automatically break the circular reference.

Iv. Python GC Module

With the problem of circular referencing, of course there is a workaround, and the Python GC module provides a mechanism that allows us to access those objects that define the __del__ (self): the function has a circular reference.

classA:def __init__(self): self.t=NonePrint 'new obj, ID is%s'%Str (hex ( self))def __del__(self):Print 'del obj, id ID%s'%Str (hex ( self))deff (): C1=A () C2=A () c1.t=C2 c2.t=C1delC1delC2PrintGc.collect ()#explicitly perform garbage collection    PrintGc.garbageif __name__=='__main__': Gc.set_debug (GC. debug_uncollectable)#set up logs for GC modulesF ()

Gc.set_debug (GC. debug_uncollectable) Set GC module after performing the cleanup, put the Uncollectable object in Gc.garbage, the above code runs the result:

 is 0x2b8f148l are 0x2b8f3c8l4[<__main__. A instance at 0x0000000002b8f148>, <__main__. A instance at 0x0000000002b8f3c8>]

We created two objects (0x2aff148l, 0x2aff3c8l) after a circular reference and after Del, are placed in the gc.garbage, so they are all uncollectable types of objects, we remove the circular reference, we can traverse the Gc.garbage object, and the object in the Garbage Object del drop.

V. Three generations of Python recycling

In Python, the method of generational collection is used. The object is divided into three generations, the beginning, the object at the time of creation, placed in a generation, if in a generation of garbage inspection, the change to survive, will be placed in the second generation, the same time in a second generation of garbage inspection, the object survived, will be placed in three generations.

The GC module will have a counter with a 3-length list, which can be obtained by Gc.get_count ().

For example (488,3,0), where 488 is the distance from the last generation of garbage checks, the number of Python allocated memory minus the number of freed memory, note that the memory allocation, rather than the increase in the reference count. For example:

Print Gc.get_count () # (590, 8, 0) A = ClassA () print Gc.get_count () # (591, 8, 0) del aprint gc.get_count () # (590, 8, 0)

3 refers to the last second generation of garbage inspection, a generation of garbage inspection, the same, 0 is the distance from the last three generations of garbage inspection, the second generation of garbage check the number of times.

GC modulo fast has a threshold for automatic garbage collection, that is, a tuple of length 3 obtained through the Gc.get_threshold function, for example (700,10,10)

Each time the counter increases, the GC module checks to see if the added count reaches the threshold number, and if it does, it executes the corresponding algebraic garbage check and resets the counter

For example, suppose the threshold is (700,10,10):

• When the counter is increased from (699,3,0) to (700,3,0), the GC module executes gc.collect (0), which checks the garbage of a generation object and resets the counter to (0,4,0)
• When the counter is increased from (699,9,0) to (700,9,0), the GC module executes gc.collect (1), which checks the garbage of the one or two-generation object and resets the counter to (0,0,1)
• When the counter is increased from (699,9,9) to (700,9,9), the GC module executes Gc.collect (2), that is, check the garbage of one or two, three generations of objects, and reset the counter to (0,0,0)

Vi.. Application

• Avoid circular references in projects
• Introduction of GC module to initiate automatic cleanup of GC module object mechanism of circular reference
• Due to generational collection, centralized management of variables that require long-term use, and moving to the second generation as soon as possible, reducing the consumption of GC checks
• The only thing that the GC module cannot handle is that the __del__ method is used in the circular reference class, so the project avoids defining the __del__ method, and if it is necessary to use the method and causes a circular reference, the code must explicitly invoke the __del__ of the object inside the gc.garbage to break the deadlock.

Reference Document: Https://docs.python.org/2.7/library/gc.html

Python garbage collection mechanism