Win32 debug CRT heap Internals

Win32 debug CRT heap Internals

... By Andrew birkett (andy@nobugs.org)

If you are lazy, skip the explanation and jump to the table at the bottom of the page

When you compile programs with devstudio in debug mode, all of your callto malloc () and free () use a special "debug" implementation. rather than being blazingly fast, the debug heap concerns itself with spotting heap errors. it achieves this by surrounding your memory blocks with guard bytes (aka "no mans land", 0xfd) so that it can detect buffer overruns and underruns. it also initialises newly allocated memory to a fixed value (0xcd) to aid reproducability. finally, it sets free () d blocks to a known value (0xdd) so that it can detect that people are writing through dangling pointers.

Mnemonics for remembering what each fill-pattern means:

• The newly allocated memory (0xCD) isCLean memory.
• The free () d memory (0xDD) isDEAD memory.
• The guard bytes (0xFD) are likeFEnces around your memory.

The debug CRT heap defers most of the heavy work to the Win32 heap functions heapalloc () and heapfree (). therefore, you won't see any first-fit or "buddy system" code in the debug CRT. the 4 GB virtual memory space which you process has is sliced up and managed by the Win32 heap inside kernel32.dll.

If you call malloc (8), the debug CRT will request a 48 byte block from heapalloc (). it needs the extra 40 bytes to store information about the memory blocks-such as the file/line where malloc () was called, and space for links to the next/Prev heap block. in the table below, all of the debug CRT information is colored in shades of red.

Heapalloc () itself needs bookkeeping information. the heapalloc (40) Call will, in turn, reserve a total of 80 bytes from your process's address space. eight bytes of bookkeeping appear below the requested 40 bytes, And the other 32 bytes appear abve it. in the table below, the Win32 heap bookkeeping is colored gray. the memory which you get back from heapalloc () is always initialised to the 4 byte pattern 0xbaadf00d.

(As an aside, When you request pages from the VM Manager via virtualalloc, they are initialized to zero, so heapalloc is actually doing additional work to re-Initialize them to this pattern ).

Once the debug CRT has got it's 40 byte block, it will fill in it's book-keeping information. the first 2 words are links to the previous and next blocks on the CRT heap. the choice of names is confusing, because the "Next" pointer actually takes you the block which was allocatedBeforeThis one chronologically, while the "previous" pointer takes you to the one allocatedSubsequently. The reason for the naming is that the linked list starts at the last-allocated block, and progresses back in time as you follow "Next" links. the debug CRT heap also internally maintains pointers to the first and last blocks (_ pfirstblock and _ plastblock) to allow heap-Checking code to traverse all the blocks.

If the filename and line number of the malloc () call are known, they are stored in the next 2 words. following that, the next word tells you how to specify bytes were requested. the next word gives a type field. usually this is "1" which means a normal block, allocated by malloc/New. it will be "2" for blocks allocate by the CRT for its own internal purposes, and "0" for blocks which have been freed but not released back to the Win32 heap (see below for more info ). the final word is a simple counter which increases Everytime an allocation is made.

Surrounding the 8-byte malloc () 'd memory there are areas of "no mans land ". these are filled with a known value (0xfd), and when the block is free () d, the CRT will check that they still have the right value. if they 've changed, then your program contains an error. unfortunately, the uption may have happened a long time ago. you can use purify or boundschecker to stop at the specified uption point, or If you don't fancy spending any money, you can wait a few days until I write an article telling you how to do it using only a bit of ingenuity!

The eight bytes which were originally requested are initialised with 0xcd. If you see this pattern appearing in your variables, you have forgotten to initialise something.

When you call free () on the 8-byte block, the CRT sets the whole 48-byte block (including its bookkeeping) to 0xdddddddddd. this means that it can tell if the block gets subsequently altered (ie. via a dangling pointer) by checking that the pattern is still there.

At this point, two things can happen. normally, heapfree () will be called to return the block to the Win32 heap. this causes the block to be overwritten with the Win32 heap's "freed memory" pattern, which is 0xfeeefeee. note that the debug CRT does not maintain any "free lists"-all of that is done within the black box of heapfree ().

However, you can also tell the debug heap to keep hold of free () d blocks. you do this by passing the _ crtdbg_delay_free_mem_df flag to _ crtsetdbgflag (). in this case, the debug CRT will keep hold of the block. this is useful if you are trying to track down a dangling pointer error, since memory blocks will not be reused and you shoshould have CT them to remain filled with 0 xdddddddddddd unless someone is writing to the free () D block. you can call _ crtcheckmemory () and it will tell you if any of these values have been tampered.

Here's an allocation I prepared earlier...

I called malloc (8) followed by free () and stepped through the CRT callto see how the memory was changed. read the columns from left to right, and you will see what values appear in memory at various stages during malloc () and free (). the call to malloc (8) returned a block at address 0x00321000, and I 've got ded offsets from that address so that you can find out the information for one of your allocations.

Address	Offset	After heapalloc ()	After malloc ()	During free ()	After heapfree ()	Comments
0x00320fd8	-40	Zero X 01090009	Zero X 01090009	Zero X 01090009	0x0109005a	Win32 heap info
0x00320fdc	-36	Zero X 01090009	Zero X 00180700	Zero X 01090009	Zero X 00180400	Win32 heap info
0x00320fe0	-32	0xbaadf00d	Zero X 00320798	0 xdddddddd	Zero X 00320448	PTR to next CRT heap block (allocated earlier in time)
0x00320fe4	-28	0xbaadf00d	Zero X 00000000	0 xdddddddd	Zero X 00320448	PTR to Prev CRT heap block (allocated later in time)
0x00320fe8	-24	0xbaadf00d	Zero X 00000000	0 xdddddddd	0 xfeeefeee	Filename of malloc () call
0x00320fec	-20	0xbaadf00d	Zero X 00000000	0 xdddddddd	0 xfeeefeee	Line number of malloc () call
0x00320ff0	-16	0xbaadf00d	Zero X 00000008	0 xdddddddd	0 xfeeefeee	Number of bytes to malloc ()
0x00320ff4	-12	0xbaadf00d	Zero X 00000001	0 xdddddddd	0 xfeeefeee	Type (0 = freed, 1 = normal, 2 = CRT use, etc)
0x00320ff8	-8	0xbaadf00d	Zero X 00000031	0 xdddddddd	0 xfeeefeee	Request #, increases from 0
0x00320ffc	-4	0xbaadf00d	0 xfdfdfdfd	0 xdddddddd	0 xfeeefeee	No mans land
Zero X 00321000	+ 0	0xbaadf00d	0 xcdcdcdcd	0 xdddddddd	0 xfeeefeee	The 8 bytes you wanted
Zero X 00321004	+ 4	0xbaadf00d	0 xcdcdcdcd	0 xdddddddd	0 xfeeefeee	The 8 bytes you wanted
Zero X 00321008	+ 8	0xbaadf00d	0 xfdfdfdfd	0 xdddddddd	0 xfeeefeee	No mans land
0x0032100c	+ 12	0xbaadf00d	0xbaadf00d	0 xdddddddd	0 xfeeefeee	Win32 heap allocations are rounded up to 16 bytes
Zero X 00321010	+ 16	0 xabababab	0 xabababab	0 xabababab	0 xfeeefeee	Win32 heap bookkeeping
Zero X 00321014	+ 20	0 xabababab	0 xabababab	0 xabababab	0 xfeeefeee	Win32 heap bookkeeping
Zero X 00321018	+ 24	Zero X 00000010	Zero X 00000010	Zero X 00000010	0 xfeeefeee	Win32 heap bookkeeping
0x0032101c	+ 28	Zero X 00000000	Zero X 00000000	Zero X 00000000	0 xfeeefeee	Win32 heap bookkeeping
Zero X 00321020	+ 32	Zero X 00090051	Zero X 00090051	Zero X 00090051	0 xfeeefeee	Win32 heap bookkeeping
Zero X 00321024	+ 36	0xfeee0400	0xfeee0400	0xfeee0400	0 xfeeefeee	Win32 heap bookkeeping
Zero X 00321028	+ 40	Zero X 00320400	Zero X 00320400	Zero X 00320400	0 xfeeefeee	Win32 heap bookkeeping
0x0032102c	+ 44	Zero X 00320400	Zero X 00320400	Zero X 00320400	0 xfeeefeee	Win32 heap bookkeeping

(I 've tried to helpfully color-code things. blue and gray is for Win32 heap stuff, and reds are for the debug CRT heap stuff. if you hate the color scheme, the colors are set up using CSS at the top of this. HTML file-Go edit them yourself !)

If you want to read more, check out debugging Windows programs, part of the developmentor series.

Any comments or feedback? Please email me.