Python Memory Management Analysis _python

In this paper, the Python memory management mechanism is analyzed in detail. Share to everyone for your reference. The specific analysis is as follows:

Memory management, a vital part of a dynamic language such as Python, largely determines the efficiency of Python execution, because in Python's run, a large number of objects are created and destroyed, all of which involve memory management.

Memory pool for small chunks of space

In Python, many times the requested memory is small chunks of memory, which are soon released after the application, because these memory applications are not designed to create objects, so there is no object-level memory pool mechanism.

Python Memory pool panorama

This means that Python performs a lot of malloc and free operations during the run, frequently switching between the user state and the kernel mentality, which can seriously affect the execution efficiency of Python. To speed up Python's efficiency, Python introduced a memory pool mechanism to manage application and release of small chunks of memory. This is also the Pymalloc mechanism mentioned earlier.

In Python 2.5, the default small block of memory within Python is set at 256 bytes, which is controlled by the symbol named Small_request_threshold that we saw earlier.

That is, when the requested memory is less than 256 bytes, Pyobject_malloc requests memory in the memory pool, and when the requested memory is greater than 256 bytes, the pyobject_malloc behavior is reduced to the malloc behavior. Of course, by modifying the Python source code, we can change this default value to change Python's default memory management behavior.

When the reference count of an object is reduced to 0 o'clock, the destructor corresponding to the object is invoked.

But to be particularly aware of, calling a destructor does not mean that it will eventually call free to release the memory space, and if so, the frequent application and free memory space will make Python's execution much less efficient (and python has been burdened with the efficiency of its execution for years). In general, Python uses a large number of memory object pooling techniques that can be used to avoid the frequent application and freeing of memory space. As a result, in the destructor, the space occupied by the object is usually returned to the memory pool.

"The problem is that Python's arena never releases pool." Why does this issue cause similar memory leaks? In this case, the request 10*1024*1024 a small 16-byte memory, which means that 160M of memory must be used, since Python does not default to open the previously mentioned With_memory_limits compilation symbol for the restricted memory pool. So Python will completely use arena to meet your needs, which is all right, and the key problem is that after a while you're releasing all these 16 bytes of memory, all of which are back to arena control, which seems to be fine.

But the problem just happened. Because Arena will never release the pool collection it maintains, so this 160M of memory is always occupied by Python, if the program in the future will not need to run the 160M such huge memory, this memory is not wasted? "

Python memory management rules: When Del, the elements of the list are released and the large objects of the management elements are recycled into the Py object buffer pool.

I hope this article will help you with your Python programming.