Today, there are a number of 32-bit applications that feel that the virtual memory available on 32-bit platforms is limited, for program developers, even when you start to focus on the 64-bit platform, you have to maintain a 32-bit version of the software, which requires a way to keep all two versions of your code fairly scalable.
Current memory Profiling Tools can help determine, what happens when the program reaches peak memory usage, but these tools are too focused on the allocated block of memory, not the committed virtual memory address space, and there is no direct correlation between the two metrics, such as memory leaks, memory fragmentation, A waste of space within a block of memory, or an overly-deferred memory cell reallocation, can result in unnecessary virtual memory submissions. Runtime Profiling tools such as IBM Rational purify or Parasoft inuse can provide a description of memory leaks and used memory, which is useful, but a particular block of memory may or may not affect virtual memory coverage, and Even a small block in a fragmented memory heap can directly affect the virtual memory footprint. On the other hand, any block of memory in this range--or even a leaking block, for virtual memory coverage, it doesn't matter, unless every useful chunk of memory in this range is reassigned to a more compact range, which is a bit like a garbage collection mechanism for Java or hosted programs, but for most C/ C + + local applications, it is absolutely impossible, because in the virtual memory space, the location of their memory block is indeterminate.
As for local code, unnecessary virtual memory usage, this actual problem is more substantive than the theoretical problem of the clean memory block. An open memory block can lead to waste of virtual memory, resulting in excessive overhead, or not, which depends entirely on whether the heap manager commits more virtual memory to support this waste. Some very small unused blocks of memory do not cause unnecessary heap "extensions". Rather than letting you guess which or how many wasted chunks of memory are causing the heap to expand, learn how to determine what a meaningful waste is. When the heap contains blocks of memory that are no longer in use, then by adding checks on the unchecked heap, you can identify the necessary memory block cleanup that has much to do with your program's virtual memory requirements.
to find out which chunks of memory you need to pay more attention to, you must add some extra code to the program to track the memory heap scope and allocated memory blocks. It might be a good idea to compile the extra code and generate a specific version.
To do this, you need to write a custom memory allocation routine, track each memory block, have a custom release routine, and track the location of the heap in virtual memory, see Example 1 and Example 2 of the pseudo code algorithm. You may also need to write custom access functions to mark the blocks of memory that you have visited so that you can release virtual memory at the right time, all of which do not require excessive memory overhead. On the other hand, if your program uses a lot of memory in the form of a heap, it will greatly degrade performance, and the method here is not a long-term solution.
Case 1:
/* Input Parameters * *
Address Triggeraddr
SIZE triggersize
List A list of tracked heap ranges
IF (the virtual memory at Triggeraddr. Tracked on the list as part of a heap range)
Todo
IF (triggeraddr + triggersize >
(The tracked upper boundary of that heap range))
Todo
/* An existing heap range is extended * *
Make system call (s) to determine the base and extent of the newly committed range which contains the addresses from trigger Addr to (triggeraddr + triggersize)
Update the size of the tracked heap range to indicate its new upper limit
End
End
ELSE do
* * There is a new heap range in the triggeraddr.
Make system call (s) to determine the base and extent of the newly committed range which contains the addresses from trigger Addr to (triggeraddr + triggersize)
Track the new committed range in the list of heap ranges
End |
Case 2:
/* input parameter */
address triggeraddr
SIZE triggersize
L IST a list of tracked heap ranges
/* local variable */
Address origrangebase
SIZE origrangesize
BOOL bfoundchange
BF Oundchange = FALSE
IF (the virtual memory at TRIGGERADDR was not tracked on the heap range list as part of a heap range)
Do
/* It seems that we are clear about this release. */
End
ELSE IF (A access exception occurs when memory at TRIGGERADDR are read)
do
Bfoundchange = TRUE
endif (bfoundchange) do
/* The amount of virtual memory that the heap consumes is changed because the space occupied by the previous memory block is freed. */
Make system calls to determine the bases and extents the tracked committed heap ranges in the immediate Of the decommitted range that includes the addresses from TRIGGERADDR to (triggeraddr + triggersize)
/* Update heap range tracking to reflect the remaining submitted Scope */
IF (any portion of the tracked heap range, contained the block at TRIGGERADDR was still committed)
do
Up Date the heap range list to track just the portion (s) of thAt range this remain committed
End
ELSE
Does
Delete the list element that tracks the range
End
End |
Trace heap memory block
can use a custom memory allocation function for memory block tracking, which is originally called a normal memory allocation function, for example, the C language program generally uses malloc (), and thereafter, the custom memory allocation will do the following series of operations:
the newly allocated memory block in the list of currently allocated memory blocks.
Determines whether virtual memory is submitted to the system.
If virtual memory has been committed, track the heap scope that contains this memory block and update the list of heap memory blocks to identify blocks of memory that have never been accessed.
also requires custom release and reallocation memory functions to update the list of memory blocks by the address and size of the memory blocks used in the program. The list of heap memory blocks to be tracked should contain the following structure:
The base address of the memory block.
Own Size
is used to indicate whether a block of memory has been accessed since the last time virtual memory was committed.
when a block of memory is freed, the custom release code will do the following:
If the memory block has not been accessed since the last heap extension, it will be reported to the program.
removes the tracked memory block from the list.
Determines whether the system has freed virtual memory that contains this block of memory.
If virtual memory is released, update it accordingly to reflect the remaining heap scope.
when a chunk of memory is reassigned, your custom reallocation memory code must do the following two things: first, after the release of the memory block, because the reallocation of memory blocks may not be the original location, and secondly, after the allocation of the new base address and allocate memory block size. Alternatively, you can check if the memory blocks that were reassigned are moved, if it is not moved, simply update the memory block tracking list, mark the size of the memory block, and if the memory block is still on the same base address, but grow, check the heap extension and start over with the method of allocating memory as described above.
Tracking Heap itself
The heap trace depends on whether virtual memory is committed or released when the memory block is allocated or released, in order to establish a heap memory range tracking table to determine the exact location of the heap in the virtual memory space during the program's operation, and the following data should be included in the tracking list:
The base address of the tracking range
Own Size
in the Windows operating system, these values are available through HeapWalk () calls, and it is important to note that HeapWalk () function calls are expensive, so they are invoked only when the program needs them, not when there is memory allocation or release. Another method on Windows is to use the Isbadreadpointer () function, and when a block of memory is freed, you can call this function to quickly determine whether the virtual memory containing the memory block has been freed. Another alternative that can be cross-platform is to try to access the virtual memory that contains this block of memory and catch possible access exceptions. In addition, it is only in one case that the expensive function such as heapwalk () is considered to determine the adjacent remaining committed heap scope.
a heap memory tracking list continues to grow through an algorithm that detects heap memory submissions, as shown in Example 1. Note that when your program allocates a block of memory, the custom memory allocation code can also track the memory blocks and use the algorithm in Example 1 to update the heap list containing the memory blocks. If the allocation of a block of memory causes additional virtual memory to be committed, the virtual memory consumed by the memory block is released before, and may be used for other purposes. In either case, the heap scope tracking list must be conditionally updated: L If you are tracking the base address of a committed range, you must update the scope size to indicate a new range upper limit.
Otherwise, a new heap memory range tracking table must be established.
If a new block of memory is present in a heap range that has not been tracked before, the above conditions are met, and it is wise to use the aforementioned system calls to efficiently track the heap memory range.
When your program frees up a block of memory, the custom memory release code uses the algorithm in Example 2, this algorithm first determines whether the freed memory is related to the heap memory range being tracked; Next, you must check that the freed memory block is still in the committed state, and if so, it indicates that even if the memory block is released, Virtual memory coverage has not changed either, otherwise your code must make a system call at the end of example 2-such as HeapWalk () in Windows-to ensure that the tracked heap is up to date and that virtual memory containing the heap memory block has been freed.
If virtual memory has been committed, then you should check the memory range that typically contains the most recently released heap memory block. To verify that the memory in this range is committed, and that any part of the memory being committed in this range should be within a heap memory range, especially if it contains a block of memory on the tracking list , this check ensures further accuracy. Here are two things to follow, as shown in Example 2:
If any part of the heap memory being tracked is committed, you must update your tracking list.
Otherwise, delete the elements in the list and track the new range.
if the portion of the submission is in the middle, then it is possible to cut the heap memory range into several breaks, and in short, before you give up tracking the old range, you should check the entire scope of which part is still in the commit state.
Several different heaps may be used in the
program, and on Windows, if you call HeapCreate () and pass the returned handle to the next HeapAlloc (), HeapReAlloc (), HeapFree () function call, A few different heaps are created, and if you load multiple instances of the C run-time Library DLL, multiple heaps will be generated because each instance uses its own heap. At this point, you can track multiple heaps in a different list, or you can track their memory blocks in different lists. First of all, the advantage of this is that the lookup of the list can become very fast, and secondly, when the heap is "destroyed", you can clear the list of traces without any hesitation, but this requires adding extra code to the heap creation and release-that is, to set up and delete the corresponding list separately. Alternatively, you can manage a list of heap memory blocks and a list of heap scopes, and build them as the program begins to run.
Some professional run-time analysis tools can also track allocated memory blocks and heap ranges, just as the previously defined memory allocation function, like the release function, can even be tracked further by a basic virtual memory hash value (ox187d690). To provide a more reliable means for accurate run-time error detection. But the method described here is not enough to help you understand when you can find a block of heap memory that can be controlled by the program and reduce the chance of virtual memory consumption.
find the right time to clean up
to use trace information to pinpoint the heap memory blocks that are causing unwanted growth in virtual memory, you must also record the memory access action and mark the corresponding memory block tracking structure when the program reads and writes the heap memory. If all of your heap memory blocks are accessed through the Access function, it is easy to find all the parts of the memory that the code writes to and generate a specific version with conditional compilation. These access functions look for blocks of memory accessed on your list and set a Boolean value to mark.
when virtual memory is submitted to generate a heap, your custom memory allocation function should suppress all memory blocks in the heap, and in the next, they may be flagged again, one after another, just as your program is accessing them. When a unmark block of memory is released, the corresponding virtual memory is also released, and your custom release function will first release the memory block to reduce the virtual memory coverage.
can also schedule some temporary scans of the heap memory blocks, which may be done at the time of each virtual memory commit. If a block of memory remains unlabeled after multiple scans (which may take a long time), the virtual memory range that contains the memory block must be checked for the number of blocks of memory being tracked. If that chunk of memory, or a group of neglected similar chunks of memory, only has a separate relationship with the committed dummy, you may have found a good way to reduce the virtual memory requirements of the program by releasing and relocating the memory blocks.
If you implement all of the memory blocks and heap range tracking codes described here, and when all of these traces work, then the speed of the program will become very slow, mainly because every time the heap memory access, will be a list lookup, of course, also through some quick list search methods such as the binary search, Jump lookup and so on to shorten the time to find, you can also use a separate list of each heap to speed up the lookup. If the program uses a lot of heap memory blocks and also finds ways to reduce the amount of extra virtual memory consumed, all the energy and patience that has been spent in the past is nothing compared to the rewards of this time.
