Heap Management in Windows Programming

Abstract: This article mainly discusses the heap Management Technology in Windows Memory Management, it also briefly introduces heap creation, memory block allocation and redistribution, heap revocation, and use of new and delete operators.

Key words: heap; heap Management
1 Introduction

In most Windows ApplicationsProgramIn the design, memory operations and management are almost inevitable. It is particularly important to dynamically allocate large-size memory. This article mainly discusses the heap Management Technology in memory management.

Heap is actually an area in the reserved virtual address space. At the beginning, most pages in the reserved area were not submitted to the physical storage. As more and more memory allocations are made from the heap, the heap manager will gradually submit more physical memory to the heap. The physical memory of the heap is allocated from the system page file. When it is released, a dedicated heap manager is responsible for recycling the occupied physical memory. Heap management is also a memory management mechanism provided by windows. It is mainly used to allocate small data blocks. Compared with the other two memory management mechanisms of windows, virtual memory and memory ing files, the heap does not have to consider the cumbersome and easy-to-ignore problems such as system allocation granularity and page boundaries, focus on program functionsCodeDesign. However, using heap to allocate and release memory is much slower than the other two machine systems, and it does not have the ability to directly control the submission and recovery of physical memory.

At the beginning of the process, the system creates a heap in the virtual address space of the newly created process. The heap is the default heap of the process. The default size is 1 MB, this value can be changed when linking the program. The default heap of a process is important and can be used by many Windows functions. During use, the system must ensure that only one thread can allocate and release memory blocks in the default heap within the specified time. Although this restriction will have a certain impact on the access speed, it can ensure the sequential access to the default heap when multiple threads in the process call various Windows functions at the same time. Multiple stacks are allowed in the process. Each heap, including the default heap, has a heap handle to identify the heap. Unlike the self-created heap, the creation and destruction of the Process's default heap are all completed by the system, and its life cycle has started before the process starts to execute, although the getprocessheap () function can be used in the program to obtain the process's mapper handle, it is not allowed to call the heapdestroy () function to undo it explicitly.

2 Requirements for dynamic heap Creation

As mentioned above, apart from the default process heap, you can also dynamically create independent heap in the virtual address space of the process. When designing a program, you do not need to dynamically create an independent heap. You can choose whether there is a need to protect the components, whether the memory can be managed more effectively, and whether local access is required. needs, whether there is a need to reduce thread synchronization overhead, and whether there is a need to quickly release the heap, etc.

This principle is easy to understand if there is a need to protect components. In Figure 1, the picture on the left shows the use of a heap by a linked list (node structure) component and a tree (branch structure) component. In this case, because the data of the two components is mixed in the heap, if the last few bytes of Node 3 (which belongs to the linked list component) are incorrectly rewritten, it may affect branch 2 (belonging to the tree component) located after it ). This will cause the relevant code of the tree component to be unable to be carried out due to memory corruption when traversing its tree. The reason is that the memory of the tree component is caused by incorrect operations on the linked list. If the tree component and the linked list component are stored in an independent heap, the preceding situation does not occur obviously, errors are limited to linked list components with incorrect operations, and tree components are protected because they are stored in independent stacks.

 
Figure 1 Role of dynamic heap creation in protection components

In, if each node of the linked list component occupies 12 bytes, and each tree component's branch occupies 16 bytes, if these objects of different lengths share one heap (left ), in the left figure, the memory allocated objects are full of the heap. If node 2 and node 4 are released, 24 bytes of fragments are generated, if you try to allocate a 16-byte branch object within the free interval of 24 bytes, the allocation fails even though the number of bytes to be allocated is smaller than the number of idle bytes. Only objects of the same size can be allocated in the stack for more effective memory management. If you change the tree component to another 12-byte group, after you release an object, the other object can be filled in the space of the newly released object.

The need for local access is also an important principle. The system usually exchanges pages between memory and system page files. However, if too many page files are exchanged, the system performance will be greatly affected. Therefore, when designing a program, avoid frequent page exchange by the system. If the data that will be accessed at the same time is distributed in close proximity, this will reduce the frequency of page exchanges between the system's memory and page files.

Thread Synchronization overhead refers to the overhead that is required to execute additional code to protect data from being damaged when multiple threads attempt to access the data simultaneously by default. This overhead ensures the thread security of the heap, so it is necessary. However, for a large number of heap Allocation Operations, this overhead will become a burden and reduce the running performance of the program. To avoid this extra overhead, You can notify the system to only access a single thread when creating a new heap. In this case, applications are responsible for the thread security of the heap.

Finally, if you need to quickly release the heap, you can use the dedicated heap for some data structures and release the whole heap without explicitly releasing each memory block allocated in the heap. For most applications, such processing can run at a faster speed.

3. Create a heap

In the process, you can dynamically create a heap based on the original default heap, which can be completed by the heapcreate () function:

Handle heapcreate ( DWORD floptions, DWORD dwinitialsize, DWORD dwmaximumsize );

The first parameter floptions specifies the operation attribute for the new heap. This flag affects access to new heap functions, such as heapalloc (), heapfree (), heaprealloc (), and heapsize. The possible values are the following flags and their combinations:

Attribute flag	Description
Heap_generate_exceptions	In the event of a function failure caused by memory out-of-bounds, the system throws an exception to indicate this failure, rather than simply returning a null pointer.
Heap_no_serialize	Indicating that mutex does not occur

The dwinitialsize and dwmaximumsize parameters are respectively the initial size of the heap and the maximum size of the stack. The value of dwinitialsize determines the number of bytes initially submitted to the heap. If the value is not an integer multiple of the page size, it is rounded to the adjacent page boundary. Dwmaximumsize is actually the maximum number of bytes in the address space area that the system can reserve for the heap. If the value is 0, an extensible heap will be created. The size of the heap is limited by the available memory. If the application needs to allocate large memory blocks, set this parameter to 0. If dwmaximumsize is greater than 0, this value limits the maximum value that can be created by the heap. heapcreate () also rounds the value to the adjacent page boundary, then, the virtual address space of the process is reserved as a part of the heap size. The size of the memory block allocated in this heap cannot exceed 0x7fff8. Any attempt to allocate a larger memory block will fail, even if the configured heap size is sufficient to accommodate this memory block. If heapcreate () is successfully executed, a handle that identifies the new heap is returned and can be used by other heap functions.

Note that when setting the first parameter, exercise caution when using the heap_no_serialize flag. Generally, avoid using this flag. This is related to the execution process of heapalloc (), the heapalloc () will perform the following steps when trying to allocate a memory block from the heapalloc:

1) traverse the chaintable of allocated and released memory blocks

2) Search for the address of an idle memory block

3) allocate new memory blocks by marking idle memory blocks as "allocated"

4) Add the newly allocated memory block to the memory block list

If two threads 1 and 2 attempt to allocate memory blocks from a heap at the same time, thread 1 will get the address of the space memory block after executing steps 1 and 2 above. However, due to the slice of the CPU on the thread running time, thread 1 may be snatched by thread 2 before the execution of step 1 and have the opportunity to perform the same steps 1 and 2, because the first thread 1 is not executed in step 1, thread 2 searches for the address of the same idle memory block and marks it as allocated. After thread 1 resumes running, it does not know that the memory block has been marked by thread 2. Therefore, two thread troops think that it is allocated an idle memory block, and update their joined tables. Obviously, errors such as two threads having identical memory block addresses are very serious and hard to find.

The preceding problems may occur only when multiple threads are simultaneously operated, A simple solution is to allow a single thread to exclusively access the heap_no_serialize instead of using the heap_serialize flag. To ensure security, make sure that the process is single-threaded or multi-threaded in the process, but only a single thread accesses the heap. In addition, multiple threads are used to access the heap, but these threads use some thread synchronization method. If you can ensure that one of the above items is true, you can also use the heap_no_serialize flag safely, and you will have a fast access speed. If you are not sure whether the preceding conditions are met, we recommend that you access the heap in sequence instead of using this flag, although the thread speed will decrease, it can ensure that the heap and its data are not damaged.

4. allocate memory blocks from the heap

After creating a heap, you can call the heapalloc () function to allocate memory blocks from the heap. In addition to allocating memory blocks from the dynamic heapcreate () created using heapcreate (), you can also allocate memory blocks directly from the default heap of the process. The following is a prototype of the heapcreate () function:

Lpvoid heapalloc ( Handle hheap, DWORD dwflags, DWORD dwbytes );

The hheap parameter is the heap handle from which the memory block to be allocated comes from. It can be a dynamic heap handle created from heapcreate () or a default heap handle obtained by getprocessheap. The dwflags parameter specifies the flag that affects heap allocation. This flag overwrites the corresponding flag specified when heapcreate () is called. The possible values are:

Flag	Description
Heap_generate_exceptions	This flag specifies that an exception will be thrown during operations such as out-of-memory operations, rather than simply returning a null pointer.
Heap_no_serialize	The call to heapalloc () is forcibly performed in different sequence from other threads accessing the same heap.
Heap_zero_memory	If this flag is used, the newly allocated memory content will be initialized to 0

The last parameter dwbytes sets the size of the memory block to be allocated from the heap. If heapalloc () is successfully executed, the address of the memory block allocated from the heap is returned. If the heapalloc () function fails to be executed due to insufficient memory or other causes, an exception is thrown. The cause of memory allocation failure can be obtained through the exception flag: If it is status_no_memory, it indicates it is caused by insufficient memory; if status_access_violation is used, the attempt to allocate memory blocks fails because the heap is damaged or the function parameters are incorrect. The preceding exception occurs only when the heap_generate_exceptions flag is specified. If this flag is not specified, the heapalloc () function simply returns a null pointer when a class-like error occurs.

When setting the dwflags parameter, if the heap_generate_exceptions flag is specified when heapcreate () is used to create a heap_generate_exceptions, you do not have to set the heap_generate_exceptions flag, because the heap_generate_exceptions flag has notified the heap that an exception will occur when the memory block cannot be allocated. In addition, exercise caution when setting the heap_no_serialize flag, similar to using the heap_no_serialize flag in the heapcreate () function. If other threads use the same heap at the same time, the heap will be damaged. If the memory block is allocated in the default heap of the Process, disable this flag.

When using heapalloc (), note that the heap is used in memory management to allocate small data blocks. If the memory block to be allocated is about 1 MB, instead of using the heap to manage the memory, you should choose the memory management mechanism of the virtual memory.

5. reallocate memory blocks

During program design, the memory block size allocated in the heap is often not suitable due to lack of foresight at the beginning (in most cases, the memory allocated at the beginning is small, later, you actually need to copy more data to the memory block.) This requires you to adjust the size of the memory block after it is allocated. Heaprealloc () is a heaprealloc function. Its prototype is:

Lpvoid heaprealloc ( Handle hheap,  DWORD dwflags, Lpvoid lpmem,  DWORD dwbytes  );

The hheap parameter is the heap handle that contains the memory block to be adjusted. The dwflags parameter specifies the ID used by the heaprealloc () function when the memory block size is changed. The possible values are heap_generate_exceptions, heap_no_serialize, heap_realloc_in_place_only, and heap_zero_memory. The functions of the first two flags are the same as those in heapalloc. Heap_realloc_in_place_only indicates that the memory block in the heap is not moved when the memory block is increased. If the heaprealloc () flag is not set, heaprealloc () the function may move the original memory block to a new address. Obviously, when this flag is set to prohibit the adjustment of the first memory address, there may be insufficient memory for the memory block to be increased. In this case, the function fails to adjust the memory block size. The memory block size and position remain unchanged. The usage of the heap_zero_memory mark is slightly different. If the memory size is adjusted faster than before, the newly added memory will be initialized to 0. If the memory block is reduced after the adjustment, this sign will not work.

The last two parameters of the function, lpmem and dwbytes, are the pointer to the reallocated memory block and the number of reallocated bytes. If the function is successfully executed, the address of the new memory block that has changed the size is returned. If the heap_realloc_in_place_only flag is used during the call, the returned address will be the same as the original memory block address. If the function fails to be executed because of insufficient memory or other reasons, the function returns a null pointer. However, if heaprealloc () fails to be executed, the original memory block remains unchanged. You can use the heapsize () function to retrieve the actual size of memory blocks.

6. release heap memory and cancel heap

You can use heapfree () to release memory blocks in the heap. The function has a simple structure and only contains three parameters:

Bool heapfree ( Handle hheap, DWORD dwflags,  Lpvoid lpmem );

Here, hheap is the handle of the heap whose memory block is to be released. The dwflags parameter can be set to 0 or heap_no_serialize; the last parameter lpmem is the pointer to the memory block. If the function is successfully executed, the specified memory block is released and true is returned. The main function of this function is to help the heap manager reclaim some unused thing memory to free up more free space, but it cannot be guaranteed that it will succeed.

Finally, you can call the heapdestory () function to destroy the heap created before the program exits or the application no longer needs it. This function only contains one parameter-the heap handle to be destroyed. If heapdestory () is successfully executed, all the memory blocks contained in the heap can be released, you can also return all the physical memory and reserved address space occupied by the heap to the system and return true. This function only works for the heapcreate () heap that is explicitly created, but cannot destroy the default heap of the process. If it is forcibly created by getprocessheap () heapdestory () is called as a parameter using the default heap handle. The system ignores the call to this function.

7. overload of the New and delete Operators

The new and delete memory space dynamic allocation operators are a common method for memory management using heap in C ++, you can use these two operators to create or delete heap objects as needed. The new operator allocates a memory block of sufficient size in the heap to store objects of the specified type. If the object types Constructed each time are different, you need to allocate the space occupied by the maximum object. After the new operator is successfully executed, it returns a pointer that matches the type of the new object. If it does not match, it must be forced type conversion; otherwise, compilation errors may occur. When you no longer need this object, you must explicitly call the delete operator to release the space. This is very important. If the delete operator is not explicitly called before another object is constructed in the pre-allocated buffer or before the buffer is released, the program will have unpredictable consequences. Note the following when using the delete OPERATOR: 

1) it must use a pointer returned by the new operator.

2) This operator also applies to null pointers.

3) only one pair of square brackets are used before the pointer name. No matter the dimension of the deleted array, any number in square brackets is ignored.

class cvmshow { PRIVATE: static handle m_sheap; static int m_sallocedinheap; public: lpvoid operator new (size_t size); void operator Delete (lpvoid pvoid); }< P> ...... handle m_sheap = NULL; int m_sallocedinheap = 0; lpvoid cvmshow: Operator new (size_t size) {< br> If (m_sheap = NULL) m_sheap = heapcreate (heap_generate_tions, 0, 0); lpvoid pvoid = heapalloc (m_sheap, 0, size); If (pvoid = NULL) return NULL; m_sallocedinheap ++; return pvoid; }< br> void cvmshow: Operator Delete (lpvoid pvoid) {< br> If (heapfree (m_sheap, 0, pvoid )) m_sallocedinheap --; If (m_sallocedinheap = 0) {< br> If (heapdestory (m_sheap) m_sheap = NULL; }< BR >}

In addition to using new and delete to create and delete heap objects, you can also easily use the number of stack functions by reloading the new and delete operators for the c ++ class. The code above performs simple Overloading on them, the static variables m_sheap and m_sallocedinheap are used to share the unique heap handle among all instances of the cvmshow class (because all instances of the cvmshow class are allocated in the same heap) and the Count of allocated class objects. These two static variables are initialized as null pointers and 0 counts at the beginning of code execution.

When the new operator is called for the first time, because the static variable m_sheap is null, it indicates that the heap has not been created yet, and a heap is created through the heapcreate () function and the heap handle is returned to m_sheap. Then, the memory is allocated in the heap according to the size specified by the entry parameter size, and the memory block counter m_sallocedinheap is allocated. During subsequent calls of this operator, the heap is no longer created because the heap has been created, instead, allocate memory blocks of the specified size in the heap and count the number of allocated memory blocks.

When a cvmshow class object is no longer used by an application, you need to undo it. This is done by the overloaded Delete operator. The delete operator only accepts one lpvoid parameter, that is, the address of the deleted object. During execution, this function first calls the heapfree () function to release the specified allocated memory objects and decrease the allocated memory count by 1. If the count is not zero, it indicates that not all memory blocks in the current heap are released. If m_sallocedinheap count is reduced to 0, all cvmshow objects have been released in the heap. You can call the heapdestory () function to destroy the heap and set the heap handle m_sheap to a null pointer. It is necessary to set the heap handle to a null pointer after the heap is removed. If you do not execute this operation, when the program calls the new operator to allocate a cvmshow class object again, it will think that the heap exists and try to allocate memory in the withdrawn heap, it will obviously cause failure.

Classes designed like cvmshow are reloaded by the new and delete operators and allocated to all cvmshow class objects in a heap, you can save the heap allocation overhead and memory created for each class. Such processing also enables each class to have its own heap and allows the dispatch class to share with it. This is also a good way to deal with it in programming.

8. Summary

When using the heap, the system operation may slow down, which is usually caused by the following reasons: the distribution operation slows down; the release operation slows down; slow speed caused by heap competition; slow speed caused by heap damage; slow speed caused by frequent allocation and redistribution. Among them, competition is caused by slow speed in the allocation and release operations. For the above reasons, we recommend that you do not use the heap too frequently in the program. The code described in this article is compiled by Microsoft Visual C ++ 2000 under Windows 6.0 professional

From: http://www.chinavideo.org/index.php? Option = com_content & task = view & id = 292 & Itemid = 5