Talking about the garbage collection mechanism of Python _python

I. Garbage collection mechanism

The garbage collection in Python is based on reference counting, supplemented by generational collection. The flaw in reference counting is a circular reference problem.
In Python, the object's memory is reclaimed if the reference number of an object is a 0,python virtual machine.

#encoding =utf-8
__author__ = ' kevinlu1010@qq.com '
 
class ClassA ():
  def __init__ (self):
    print ' object born,id:%s '%str (Hex (ID (self))
  def __del__ (self):
    print ' Object del,id:%s '%str (Hex (ID (self))
 
def F1 () : While
  True:
    c1=classa ()
    del C1

The execution of F1 () loops out such results, and the memory consumed by the process is basically unchanged

Object born,id:0x237cf58
Object del,id:0x237cf58

C1=classa () Creates an object that is placed in 0x237cf58 memory, where the C1 variable points to this memory, at which point the reference count of this memory is 1
After Del C1, the C1 variable no longer points to 0x237cf58 memory, so the reference count of this memory is reduced by one, equal to 0, so the object is destroyed and the memory is freed.

Causes a reference count of +1
object is created, such as a=23
object is referenced, such as B=a
object is passed into a function as an argument, such as Func (a)
Object as an element, stored in a container, such as List1=[a,a]
Result in reference count-1
An object's alias is explicitly destroyed, such as Del A
The alias of the object is given a new object, such as a=24
An object leaves its scope, for example, when the F function completes, the local variable in the Func function (the global variable does not)
The container in which the object is located is destroyed or the object is deleted from the container

Demo

def func (c,d):
  print ' in Func function ', Sys.getrefcount (c)-1
 
 
print ' init ', Sys.getrefcount (one)-1
a = 11
   print ' after A=11 ', Sys.getrefcount (one)-1
b = A
print ' after B=1 ', Sys.getrefcount (one)-1
func (one)
p Rint ' after func (a) ', Sys.getrefcount (one)-1
list1 = [A, b]
print ' after list1=[a,12,14] ', Sys.getrefcount ( One)-1
a=12
print ' After A=12 ', Sys.getrefcount (one)-1
del a
print ' After del a ', Sys.getrefcount (11 -1
del b
print ' after Del b ', Sys.getrefcount-1
# list1.pop (0)
# print ' After pop list1 ', sys. Getrefcount-1
del list1
print ' After Del List1 ', Sys.getrefcount (11)-1

Output:

a=11 after
b=1
-func function after
func (a) after
list1=[a,12,14]
fter a=12 after del
a del
list1 24

Question: Why does calling a function make reference count +2

View a reference count for an object
Sys.getrefcount (a) can view the reference count of a object, but it is 1 larger than the normal count, because passing a function calls into a, which makes a reference count of +1

Two. Circular references cause memory leaks

def F2 (): While
  True:
    c1=classa ()
    C2=classa ()
    c1.t=c2
    c2.t=c1
    del C1
    del C2
    

Executing f2 (), the process consumes more memory than ever before.

Object Born,id:0x237cf30
Object born,id:0x237cf58

After creating the C1,C2, 0X237CF30 (c1 corresponding memory, recorded as Memory 1), 0x237cf58 (c2 corresponding memory, recorded as Memory 2) the reference count of both memory is 1, after C1.T=C2 and C2.T=C1, the reference count of these two blocks becomes 2.

After Del C1, memory 1 of the object's reference count becomes 1, because it is not 0, so memory 1 of the object will not be destroyed, so memory 2 of the object is still 2, after Del C2, similarly, memory 1 of the object, memory 2 of the object is 1.

Although their two objects can be destroyed, they cause a memory leak because they are not recycled by the garbage collector because of circular references.

Three. Garbage collection

DEFF3 ():
  # Print Gc.collect ()
  C1=classa ()
  C2=classa ()
  c1.t=c2
  c2.t=c1
  del C1
  del C2
  print gc.garbage
  print gc.collect () #显式执行垃圾回收
  print gc.garbage
  time.sleep ()
If __name__ = = ' __main__ ':
  gc.set_debug (GC. Debug_leak) #设置gc模块的日志
  f3 ()

Output:

Python

Gc:uncollectable <classa instance at 0230e918> gc:uncollectable <classa instance at
0230e940>
GC : uncollectable <dict 0230b810>
gc:uncollectable <dict 02301ed0>
Object born,id:0x230e918
Object born,id:0x230e940

4
The objects after the garbage collection are placed in the Gc.garbage list.
Gc.collect () returns the number of objects that are unreachable, 4 equals two objects and their corresponding dict

There are three scenarios that trigger garbage collection:

1. Call Gc.collect (),
2. When the counter of the GC module reaches the valve value.
3. When the program exits

Four. GC Module common function analysis

Garbage Collector interface


The GC module provides an interface for developers to set up garbage collection options. As mentioned above, one of the drawbacks of using reference counting to manage memory is circular references, and one of the main functions of the GC module is to solve the problem of circular references.

Common functions:

Gc.set_debug (Flags)
Set the debug log for the GC, which is generally set to GC. Debug_leak

Gc.collect ([generation])
Explicitly garbage collection, you can enter parameters, 0 representatives only check the first generation of objects, 1 representatives to check a, second-generation objects, 2 to check one, two, three generations of objects, if not pass parameters, execute a full collection, that is equal to pass 2.
Returns the number of unreachable (Unreachable objects) objects

Gc.set_threshold (threshold0[, threshold1[, Threshold2])
Sets the frequency at which garbage collection is automatically performed.

Gc.get_count ()
Gets the counter that is currently performing the garbage collection automatically, returning a list of length 3

Automatic garbage collection mechanism of GC module

You must import the GC module, and Is_enable () =true will start automatic garbage collection.
The main function of this mechanism is to find and handle the garbage objects that cannot be reached.
Garbage collection = garbage Check + garbage collection
In Python, the method of generational collection is used. The object is divided into three generations, at the beginning, when an object is created, it is placed in a generation, and if the object survives in a single generation of garbage checking, it is put into the second generation, and in the same time a second generation of garbage checks, the object survives and is put into three generations.

The GC module will have a counter with a 3-length list, which can be obtained by Gc.get_count ().
For example (488,3,0), where 488 refers to the last generation of garbage checks, the number of Python allocated memory minus the number of freed memory, note the memory allocation, not the increase in the reference count. For example:

Print Gc.get_count () # (590, 8, 0)
a = ClassA ()
print Gc.get_count () # (591, 8, 0)
del a
print Gc.get_coun T () # (590, 8, 0)

3 refers to the distance from the last second generation of garbage inspection, a generation of garbage check the number of times, the same, 0 refers to the distance from the last three generations of garbage inspection, the number of second generation of garbage inspection.

The GC model has a threshold for automatic garbage collection, that is, a tuple of length 3 obtained through the Gc.get_threshold function, for example (700,10,10)
Each time the counter increases, the GC module checks whether the added count reaches the number of thresholds, and if so, performs a corresponding algebraic garbage check and resets the counter

For example, suppose the valve value is (700,10,10):

When the counter is increased from (699,3,0) to (700,3,0), the GC module executes gc.collect (0), which checks the garbage of the generation object and resets the counter to (0,4,0)
When the counter is increased from (699,9,0) to (700,9,0), the GC module executes gc.collect (1), which checks the garbage of the one or two generation object and resets the counter to (0,0,1)
When the counter is increased from (699,9,9) to (700,9,9), the GC module executes Gc.collect (2), which checks the garbage of one or two, three-generation objects and resets the counter to (0,0,0)

Other

If two objects in a circular reference define the __DEL__ method, the GC module does not destroy the unreachable objects because the GC module does not know which object's __del__ method should be invoked first, so for security purposes, the GC module places the object in the Gc.garbage but does not destroy the object.

Five. Application

• Avoid circular references in projects
• Introduction of GC module to start automatic scavenging of GC module The object mechanism of the circular reference
• Because of generational collection, the variables that need to be used for a long time are centrally managed and moved to the second generation as quickly as possible, reducing the consumption of GC inspections
• The only thing that the GC module cannot handle is that a circular reference class has a __del__ method, so to avoid defining the __del__ method in the project, if you must use the method and cause a circular reference, you need code to explicitly call the __del__ of the object inside the gc.garbage to break the deadlock