Python Memory Management Analysis

In this paper, the Python memory management mechanism is analyzed in detail. Share to everyone for your reference. The specific analysis is as follows:

Memory management, which is a vital part of Python's dynamic language, largely determines the execution efficiency of Python, because a large number of objects are created and destroyed in Python's operation, which involves memory management.

Memory pools in small chunks of space

In Python, many times the requested memory is small chunks of memory, these small pieces of memory after the application, will soon be released, because these memory applications are not to create objects, so there is no object-level memory pool mechanism.

Python Memory pool panorama

This means that Python performs a large amount of malloc and free operations during runtime, frequently switching between user state and kernel mentality, which can seriously affect Python's execution efficiency. To speed up the execution of Python, Python introduces a memory pooling mechanism for managing the application and release of small chunks of memory. This is the pymalloc mechanism mentioned earlier.

In Python 2.5, Python's internal default small block memory is set at 256 bytes from the chunk memory, which is controlled by the symbol named Small_request_threshold that we saw earlier.

That is, when the requested memory is less than 256 bytes, Pyobject_malloc will request memory in the memory pool, and when the requested memory is greater than 256 bytes, the behavior of Pyobject_malloc will degenerate to the behavior of malloc. Of course, by modifying the Python source code, we can change the default value to change Python's default memory management behavior.

When the reference count of an object is reduced to 0 o'clock, the destructor corresponding to the object is called.

However, it is important to note that calling a destructor does not mean that it will eventually call free to release memory space, and if so, the frequent application and release of memory space will make Python's execution less efficient (not to mention that Python has been burdened with dissatisfaction with its execution efficiency for many years). In general, Python uses a large number of memory object pooling techniques to avoid the frequent application and release of memory space. Therefore, at the time of destruction, the space occupied by the object is usually returned to the memory pool.

"The question is: Python's arena never releases the pool. Why does this issue cause a similar memory leak? Consider a situation where applying for 10*1024*1024 16 bytes of small memory means that 160M of memory must be used, since Python does not open by default the With_memory_limits compilation symbol of the previously mentioned restricted memory pool. So Python will fully use arena to meet your needs, which is all right, the key problem is that after a while, you will all of these 16 bytes of memory are freed, the memory back to the arena control, there seems to be no problem.

But that's exactly where the problem came in. Because Arena always does not release its maintained pool collection, so this 160M of memory is always occupied by Python, if later in the program will not need 160M of such huge memory, this memory is not wasted? "

Python memory management rules: When Del, the elements of the list are freed, and the large objects of the management elements are recycled into the Py object buffer pool.

Hopefully this article will help you with Python programming.

