Python's memory management mechanism

From a lighter level, Python's memory management mechanism can be used in three ways.

(1) Garbage collection

(2) Reference count

(3) Memory pool mechanism

First, garbage collection:

Unlike languages like C++,java, Python can assign variables directly without declaring the variable type beforehand. For the Python language, the type and memory of the object are determined at run time. This is why we call the Python language dynamic type (here we can simply attribute the dynamic type to the allocation of the memory address of the variable to automatically determine the variable type at run time and assign the variable).

Second, reference count:

Python manages memory in a way similar to Windows kernel objects. Each object maintains this count of references to the object. (Image from Python core programming)

、

x = 3.14

y = X

We first created an object 3.14 and then assigned the reference to X for this floating-point object, because X is the first reference, so the reference count for this floating-point object is 1. The statement y = x creates a reference alias y that points to the same object, and we find that it does not create a new object for Y, but instead points y to the floating-point object that X points to, making its reference count 2.

We can easily prove the point above:

The ID of variable A and variable B are identical (we can think of the ID value as a pointer to a variable in c).

We cite a picture of another URL to illustrate the problem: for C, when we create a variable A, we request a memory space for that variable and put the value of the variable in that space, and when the variable is assigned to another variable B, a new memory space is applied to B, and the value of the variable is placed in the memory space of B. This is why the pointers for A and B are inconsistent.

int a = 1 int a = 2

Python is not the same, in fact, Python is handled in a somewhat similar way to JavaScript, where variables are more like tags attached to objects (similar to reference definitions). When a variable is bound to an object, the reference count of the variable is 1, (there are other cases that will also cause the variable reference count to increase), the system will automatically maintain these tags, and timed scan, when a tag's reference count becomes 0, the pair will be recycled.

A = 1 A = 2 B = A

Three, memory pool mechanism

Python's memory mechanism is the pyramid line, the -1,-2 layer mainly operates the operating system,

The No. 0 layer is the memory allocation and releasing function in C malloc,free.

1th Floor and The 2nd layer is a memory pool, Python interface function pymem_malloc function Implementation, when the object is less than 256K, the layer directly allocated memory;

The 3rd layer is the topmost, which is our direct manipulation of the Python object;

In C, there is a performance problem if you frequently call malloc and free. Coupled with frequent allocations and the release of small chunks of memory can result in memory fragmentation. Python's main work here is:

If the requested memory is used between 1~256 bytes, use its own memory management system, or use malloc directly.

This will still call malloc to allocate memory, but each time it allocates a chunk of 256k of memory.

The memory registered through the memory pool is eventually reclaimed to the memory pool and does not call C's free release. For next use. for simple Python objects, such as numeric values, strings, tuples (tuple is not allowed to be changed) in the way of replication (deep copy?), that is, when the other variable B is assigned to the variable A, although the memory space of A and B is still the same, but when the value of a is changed, Will re-allocate space for a, and the addresses of A and b become no longer the same

For a dictionary (dict), a list, and so on, changing one will cause another change, also known as a shallow copy.

Report:

Reference count increased

1. Object created: x=4

2. Other people were created: y=x

3. Passed as a parameter to the function: Foo (x)

4. As an element of the container object: a=[1,x, ' 33 ']

Reference count Reduction

1. A local reference has left its scope. For example, at the end of the foo (x) function above, X points to the object reference minus 1.

2. The alias of the object is explicitly destroyed: Del x; or del y

3. An alias of an object is assigned to another object: x=789

4. Object is removed from a Window object: Mylist.remove (x)

5. The Window object itself is destroyed: Del myList, or the Window object itself is out of scope.

Garbage collection

1. When there are no longer parts in memory, the garbage collector will clean them off. it checks for those objects that have a reference count of 0, and then clears its memory space. Of course, in addition to the reference count of 0 will be cleared, there is also a situation will be cleared by the garbage collector: when two objects refer to each other, their own other references are already 0.

2. Garbage collection mechanism There is also a recycle garbage collector, which ensures that the circular reference object is released (a reference B, B refers to a, which causes its reference count to never be 0).

Reference:

[1] Python 2.7.8 documentation Memory management

[2] In-depth explanation of Python value-transfer problem and memory management mechanism-CSDN

[3] Python Memory pool management and buffer pool design-Zhang Zhilin's Column

[4] Understanding Python variables and memory management

Python's memory management mechanism