Example of the python garbage collection mechanism, python garbage collection

Example of the python garbage collection mechanism, python garbage collection

Example of the python garbage collection mechanism

For more details about pythonn garbage collection, here is a rough summary.

In Python, reference counting is used most of the time. Each PyObject is defined as follows:

#define PyObject_HEAD          \   Py_ssize_t ob_refcnt;        \   struct _typeobject *ob_type; typedef struct _object {   PyObject_HEAD } PyObject; 

Each pyobject has a refcnt to record its own reference number. Once the reference number is 0, it is recycled.

The advantage of reference counting is the real-time performance. Once no other object is referenced, it can be immediately recycled. It looks very good, but why does not many languages use this solution, because the reference count has a fatal defect, the circular reference problem cannot be solved, for example:

a = [] b = [] a.append(b) b.append(a) 

In fact, there are no other variables that reference a and B, so they should actually be recycled. However, due to the mutual reference relationship, their reference numbers are all 1 and cannot be recycled.

In python, the problem of mutual reference only exists in the container, such as list, dictionary, class, and instance. To solve this problem, python introduces two other mechanisms: tag-clear and generational-recycle.

In fact, the containers in python are not as simple as previously mentioned. Before pyobject_head, there is also a PyGC_head, which is specifically used to handle the circular references of containers.

typedef union _gc_head {   struct {     union _gc_head *gc_next;     union _gc_head *gc_prev;     Py_ssize_t gc_refs;   } gc;   long double dummy; /* force worst-case alignment */ } PyGC_Head; 

All the objects created for the container class will be recorded in the list of objects that can be collected. Through the above structure, we can know that a two-way linked list is built, in this way, we can track all possible situations that may generate circular references. Simple containers, such as int and string, are not of the container type. As long as the reference technology is 0, they will be recycled. However, frequent malloc and free operations will seriously affect efficiency. Therefore, python uses a large number of object pools to improve efficiency.

Tag-clear includes two aspects of garbage collection: (1) Looking for recyclable objects (2) Recycling objects. The tag in python starts from the root object and traverses all container class objects, find some objects that can be reached by reference and put them in the linked list maintained by reachable. For those that cannot be reached, put them in the linked list maintained by unbreachable. After this process ends, reclaim the elements in unreachable.

So how does one correspond to the previous cycle reference? In python, a valid reference number is generated, and gc exists. in gc_refs, for example, the actual number of references of a and B is 1, but the valid number of references is 0 (the number of references in the loop is reduced by 1). Because refcnt in pyobjec cannot be directly modified, otherwise, a series of problems will occur. We can record the valid reference numbers to gc. in gc_refs, the actual valid references of a and B are all 0, so they can be recycled.

The following is another case:

a = [] b = [] c = a a.append(b) b.append(a) 

Here, AB is also a circular reference, but c is used to reference a. By calculating the valid reference count in the loop, the reference number of a is 1, and the reference number of B is 0. It seems that B should be recycled, but in fact, because a cannot be recycled, and a references B, B will also be placed in the reachable linked list and not recycled. Its gc. gc_refs will still be set to 1.

Another kind of generational recycling means that some objects in the memory will frequently malloc and free, and some will last for a long time. If an object still exists after multiple garbage collection and removal, then we can think that this object is useful for a long time and does not need to be frequently detected and recycled. Python is divided into three generations, namely three linked lists for maintenance. The zero generation can maintain a maximum of 700 objects, one for 10 objects, and two for 10 objects. If the number of objects exceeds this limit, the mark-clear algorithm will be called for recycling. It can be thought that the 0-generation objects will go to the 1-generation and 2-generation after a period of time, and then the detection and recovery of them will be less frequent than the 0-generation objects.

Note thatThe main mechanism of python is the reference technology. The mark-clearing and generational collection are only added to compensate for the disadvantages of reference counting. That is to say, the latter two can only play a role in the circular reference of the container class.

The above is an example of the python garbage collection mechanism. If you have any questions, please leave a message or go to the community on this site for discussion. Thank you for reading this article and hope to help you. Thank you for your support for this site!