Python Memory leakage and GC module usage

Python Memory leakage and GC module usage
-- Error Correction version on 6.11

Horin | He Qin
Email: horin153@msn.com
Blog:

Http://blog.csdn.net/horin153/

in python
, object reference count is used to solve the memory leakage problem and automatic garbage collection is implemented based on reference count.
because python
has the automatic garbage collection function, many beginners think they have lived a good life and do not have to be disturbed by memory leaks. However, if you look at the description of the _ del _ ()
function in the python document, you will know that there is a dark cloud in the good days. The following documents are excerpted:
some common situations that may
prevent the reference count of an object from going to zero include:
circular references between objects (e.g ., A Doubly-linked list or a
tree data structure with Parent and Child pointers ); A reference to the
object on the stack frame of a function that caught an exception (the
traceback stored in SYS. exc_traceback keeps the stack fram E alive); or a
reference to the object on the stack frame that raised an unhandled
exception in interactive mode (the traceback stored in
sys. last_traceback keeps the stack frame alive ).
it can be seen that the cyclic reference between the objects with the _ del _ ()
function is the main cause of Memory leakage. Note: python
circular references between objects without the _ del _ () function can be automatically decommissioned.

How do I know if the memory of an object is leaked?

Method 1: When you think an object should be destroyed (that is, the reference count is 0), you can use SYS. getrefcount (OBJ)
To get the reference count of the object, and determine whether the memory leakage is based on whether the returned value is 0. If the returned reference count is not 0, the object OBJ
It cannot be recycled by the garbage collector.
Method 2: You can also use the python extension module GC to view the details of objects that cannot be recycled.

First, let's look at a normal test.Code:

# --------------- Code begin --------------

#
-*-Coding: UTF-8 -*-

Import GC
Import sys

Class
Cgcleak (object ):
Def _ init _ (Self ):
Self. _ text =
'#' * 10

Def _ del _ (Self ):
Pass

Def
Make_circle_ref ():
_ Gcleak = cgcleak ()
# _ Gcleak. _ Self =
_ Gcleak # test_code_1
Print '_ gcleak ref count0: % d' %
SYS. getrefcount (_ gcleak)
Del _ gcleak
Try:
Print
'_ Gcleak ref count1: % d' % SYS. getrefcount (_ gcleak)
Except
Unboundlocalerror:
Print '_ gcleak is invalid! '

Def
Test_gcleak ():
# Enable automatic garbage collection.

GC. Enable ()
# Set the garbage collection debugging flags.

GC. set_debug (GC. debug_collectable | GC. debug_uncollectable |/

GC. debug_instances | GC. debug_objects)

Print 'in in leak
Test ...'
Make_circle_ref ()

Print 'in in collect ...'

_ Unreachable = GC. Collect ()
Print 'unreachable object num: % d' %
_ Unreachable
Print 'garbage object num: % d' % Len (GC. Garbage)

If
_ Name _ = '_ main __':
Test_gcleak ()

#---------------
Code end ----------------

In test_gcleak (), set the Garbage Collector debugging flag, and then use
Collect () is used for garbage collection, and finally the unattainable Number of spam objects found in the garbage collection and the total number of spam objects in the interpreter are printed.

GC. Garbage is a list object. The list item is an inaccessible (that is, a spam object) discovered by the garbage collector, but cannot be released (that is, it cannot be recycled. Description:
List of objects which the collector found to be unreachable but cocould
Not be freed (uncollectable objects ).
Normally, GC. Garbage
Objects in the reference ring. Because Python does not know the security order in which to call the _ del _ () function of the object in the ring, the object remains in
GC. garbage causes memory leakage. If you know a safe order, you can break the reference ring and execute del GC. Garbage [:].
To clear the spam Object List.

Output of the above Code is (# The post string is the author's note ):
#-----------------------------------------
Begin
Leak test...
# Variable _ gcleak reference count is 2.
_ Gcleak ref count0: 2
#
_ Gcleak becomes an unreachable variable.
_ Gcleak is invalid!
# Start garbage collection
Begin
Collect...
# The number of inaccessible spam objects discovered in this garbage collection is 0.
Unreachable object num: 0
#
The number of spam objects in the interpreter is 0.
Garbage object num: 0
#-----------------------------------------

It can be seen that the reference count of the _ gcleak object is correct, and no memory leakage occurs to any object.

If you do not comment out
The test_code_1 statement in make_circle_ref:
_ Gcleak. _ Self = _ gcleak
Also
It is to let _ gcleak form a loop reference for itself. Run the above Code and the output result is:
#-----------------------------------------
Begin
Leak test...
_ Gcleak ref count0: 3
_ Gcleak is invalid!
Begin
Collect...
# Garbage objects that can be recycled: The address is 012aa090 and the type is cgcleak.
GC:
Uncollectable <cgcleak 012aa090>
GC: uncollectable <dict
012ac1e0>
Unreachable object num: 2
#!! The number of garbage objects that cannot be recycled is 1, leading to memory leakage!
Garbage
Object num: 1
#-----------------------------------------
Visible
<Cgcleak 012aa090> Memory leakage of the object !! The extra dict garbage is the leaked _ gcleak
The Dictionary of the object. The output information is as follows:
{'_ Self': <__ main _. cgcleak object
0x012aa090>, '_ text ':'##########'}

In addition to its own cyclic references, cyclic references between multiple objects can also cause memory leakage. An example is as follows:

# --------------- Code begin
--------------

Class cgcleaka (object ):
Def
_ Init _ (Self ):
Self. _ text = '#' * 10

Def
_ Del _ (Self ):
Pass

Class cgcleakb (object ):
Def
_ Init _ (Self ):
Self. _ text = '* 10

Def
_ Del _ (Self ):
Pass

def make_circle_ref ():
_ A =
cgcleaka ()
_ B = cgcleakb ()
_. _ B = _ B # test_code_2
_ B. _ A = _ A # test_code_3
Print 'ref count0: A = % d B = % d' %/
(sys. getrefcount (_ A), sys. getrefcount (_ B)
# _ B. _ A = none #
test_code_4
Del _ A
Del _ B
try:
Print
'ref count1: A = % d' % sys. getrefcount (_ A)
response t unboundlocalerror:
Print '_ A is invalid! '
try:
Print 'ref count2: B = % d' %
sys. getrefcount (_ B)
response t unboundlocalerror:
Print
'_ B is invalid! '

# --------------- Code end ----------------

After this test, the output result is:
#-----------------------------------------
Begin leak
Test...
Ref count0: A = 3 B = 3
_ A is invalid!
_ B is invalid!
Begin
Collect...
GC: uncollectable <cgcleaka 012aa110>
GC:
Uncollectable <cgcleakb 012aa0b0>
GC: uncollectable <dict
012ac1e0>
GC: uncollectable <dict 012ac0c0>
Unreachable
Object num: 4
Garbage object num: 2
#-----------------------------------------
Yes
See _ A, _ B. All objects have memory leakage. Because the two are circular references, the garbage collector does not know how to recycle, that is, it does not know how to call the _ del _ () of the object first __()
Function.

Use any of the following methods to avoid Memory leakage by breaking the ring reference:
[1] Comment out make_circle_ref ()
Test_code_2 statement in;
[2] comment out the test_code_3 statement in make_circle_ref;
[3]
Uncomment the test_code_4 statement in make_circle_ref.
The output result is as follows:
#-----------------------------------------
Begin
Leak test...
Ref count0: A = 2 B = 3 # Note: The output result varies depending on the situation.
_ A is invalid!
_ B
Is invalid!
Begin collect...
Unreachable object num: 0
Garbage
Object num: 0
#-----------------------------------------

Conclusion: GC in Python has a strong function, such as setting GC. set_debug (GC. debug_leak)
You can check the memory leakage caused by cyclic references. If you check for Memory leakage during development and ensure that no memory leakage occurs during release, you can extend Python
To improve the running efficiency.

Recommended documents

1,
Python garbage collectionAlgorithmHttp://wiki.woodpecker.org.cn/moin/python_ref_circle_gc, description
2,
Garbage collection for python, http://arctrix.com/nas/python/gc/