Analyze Windows NT/2000 heap memory and Virtual Memory Organization (http://webcrazy.yeah.net)

Analysis of Windows NT/2000 heap memory and Virtual Memory Organization
WebSphere (http://webcrazy.yeah.net /)

When discussing today's topic, I think we should repeat the following two books:

Matt pietrek
<Windows 95 system programming secrets>
Jeffrey Richter
<Programming Microsoft Windows, Fourth Edition>

The heap memory and virtual memory management and organization of Windows (although not all Windows NT/2000) are described in terms of core and user States. Before reading, it is best to browse first. I will not describe some basic APIs or concepts.

First, let's talk about the heap memory organization. Some core data structures of Windows NT/2000 heap memory are stored in the process peb. The peb structure defines the following members (I'm not sure about the peb structure ):

Typedef struct _ peb
{
.
.
.
Pvoid processheap; // The default heap address of the process, located at 0x18 after peb
.
.
.
Ulong numberofheaps; // number of current workers of the Process, located at 0x88 after peb
Ulong maximumnumberofheaps; // The maximum number of heaps that a process can possess, located at 0x8c after peb
Pvoid ** processheaps; // address list of all heap owned by the process, located at 0x90 after peb
.
.
.
} Peb, * ppeb;

For the peb address and its preliminary analysis, see my organization for in-depth Windows NT/2000 modules. For better understanding, I listed some SoftICE analysis processes below:

: ADDR cmd // click the following command to check the cmd.exe process.

: Heap 32 cmd // display the heap of the CMD Process
Base id cmt/psnt/rsvd segments flags Process
00130000 01 0025/0024/00db 1 00000002 cmd
|
| _ Default heap address of a process (which is also returned to the user as a handle of the Process heap on Windows)

00230000 02 0003/0002/000d 1 00008000 cmd
007c0000 03 0008/0008/0008 1 00001002 cmd
00800000 04 0004/0004/000c 1 00001002 cmd

:? # DWORD (@ (7ffdf000 + 18) // display the default heap address of the process
00130000 0001245184 ""

: Dd 7ffdf000 + 88 L 10 // displays several heap-related members in peb.
0010: 7ffdf088 00000004 00000010 77fce380 00450000.
|
| Processheaps value. Windows 2000 provides the API getprocessheaps
| _ Used to enumerate the addresses of all heap owned by a process
| _ Maximumnumberofheaps Value
| _ Numberofheaps Value

: Dd @ (7ffdf000 + 90) l DWORD (@ (7ffdf000 + 88) * 4
0023: 77fce380 00130000 00230000 007c0000 00800000 ...... # ...... | .....
--------------------------------------
|
| _ List of all the current heap addresses of the CMD process (compare with the heap 32 command output on SoftICE)

The above analysis only roughly describes the heap. For memory usage in each heap, Windows NT/2000 provides the heapwalk API for debugging, which can traverse the content of the specified heap. The heap 32 command of SoftICE and the-W parameter can also traverse the content. In Windows NT/2000, a function family starting with the heap (rtlheap) header is provided for heap memory operations, in these functions, Windows NT/2000 eventually calls virtualalloc to implement memory usage. This is the part about virtual memory that I will discuss below. Because operations in Windows NT/2000 in the heap are very frequent, to avoid the time loss of frequent ring switching, some primary data structures of heap memory are stored in the peb structure of the User-state space, but not in the kpeb of the kernel-state space. To be familiar with the PE loading routine (LDR function family), you have to look at the Microsoft CRT source code. The CRT code sets up the CRT using heap (mainly for the new syntax) you can have a comprehensive understanding of heap applications, such as heapinit. c. heaphook. c. heapdump. c. heapwalk. c.

Next we will talk about the virtual memory organization. The basic structure of virtual memory management is VAD (virtual address descriptors ). Because the Windows NT/2000 memory subsystem actually allocates physical storage space when actually using the storage space, therefore, the kernel provides the VAD structure for storing data such as reserved virtual memory space and submission. The VAD structure is located in eprocess (kpeb:

Kd>! Processfields
Eprocess structure offsets:
.
.
.
Vadroot: 0x194
.
.
.

VAD uses a binary tree structure to organize the VAD. The position of the VAD in kpeb can be seen from the output results of the above i386kd/windbg. SoftICE's query command can export very important structures in this system. The following lists the functions that implement this command:

//-----------------------------------------------------------------
//
// Querysystemprocessvad -- portion cut from unordered ented Windows NT
// Rewrite by webcrazy (http://webcray.yeah.net) on 11-20-2000!
// Only test on Windows 2000 Server build 2195!
//
//-----------------------------------------------------------------

# Define max_vad_entries 0x200
# Define vadoffset 0x194

Typedef struct VAD {
Void * startingaddress;
Void * endingaddress;
Struct VAD * parentlink;
Struct VAD * leftlink;
Struct VAD * rightlink;
Ulong flags;
Ulong mmci;
Ulong protopte;
} Vad, * pvad;

# Pragma pack (1)
Typedef struct vadinfo {
Void * vadlocation;
VAD;
} Vadinfo, * pvadinfo;
# Pragma pack ()

Vadinfo vadinfoarray [max_vad_entries];
Int vadinfoarrayindex;
Pvad vadtreeroot;

Void _ stdcall vadtreewalk (pvad vadnode)
{
If (vadnode = NULL ){
Return;
}

Vadtreewalk (vadnode-> leftlink );

If (vadinfoarrayindex <max_vad_entries ){

Vadinfoarray [vadinfoarrayindex]. vadlocation = vadnode;
Vadinfoarray [vadinfoarrayindex]. VAD. startingaddress = vadnode-> startingaddress;
Vadinfoarray [vadinfoarrayindex]. VAD. endingaddress = vadnode-> endingaddress;

(Ulong) vadinfoarray [vadinfoarrayindex]. VAD. startingaddress <= 12;
(Ulong) vadinfoarray [vadinfoarrayindex]. VAD. endingaddress + = 1;
(Ulong) vadinfoarray [vadinfoarrayindex]. VAD. endingaddress <= 12;
(Ulong) vadinfoarray [vadinfoarrayindex]. VAD. endingaddress-= 1;

Vadinfoarray [vadinfoarrayindex]. VAD. parentlink = vadnode-> parentlink;
Vadinfoarray [vadinfoarrayindex]. VAD. leftlink = vadnode-> leftlink;
Vadinfoarray [vadinfoarrayindex]. VAD. rightlink = vadnode-> rightlink;
Vadinfoarray [vadinfoarrayindex]. VAD. Flags = vadnode-> flags;
If (vadnode-> mmci> 0x80000000)
Vadinfoarray [vadinfoarrayindex]. VAD. mmci = vadnode-> mmci;
Else vadinfoarray [vadinfoarrayindex]. VAD. mmci = 0;
If (vadnode-> protopte> 0x80000000)
Vadinfoarray [vadinfoarrayindex]. VAD. protopte = vadnode-> protopte;
Else vadinfoarray [vadinfoarrayindex]. VAD. protopte = 0;
Vadinfoarrayindex ++;

}
Vadtreewalk (vadnode-> rightlink );
}

Void vadtreedisplay ()
{
Int I;
Dbuplint ("/nvadroot is located @ % 08x/N", vadtreeroot );
Dbuplint ("VAD @ starting ending parent leftlink
Rrightlink flags mmci PTE/N ");
For (I = 0; I <vadinfoarrayindex; I ++ ){
Dbuplint ("% 08x % 08x % 08x % 08x % 08x % 08x % 08x % 08x % 08x/N ",
Vadinfoarray [I]. vadlocation,
Vadinfoarray [I]. VAD. startingaddress,
Vadinfoarray [I]. VAD. endingaddress,
Vadinfoarray [I]. VAD. parentlink,
Vadinfoarray [I]. VAD. leftlink,
Vadinfoarray [I]. VAD. rightlink,
Vadinfoarray [I]. VAD. flags,
Vadinfoarray [I]. VAD. mmci,
Vadinfoarray [I]. VAD. protopte );
}
}

Void querysystemprocessvad ()
{
If (ushort) ntbuildnumber )! = 2195 ){
Dbuplint ("only test on Windows 2000 Server build 2195! /N ");
Return;
}

// Query system process VAD
Vadtreeroot = (pvad) * (ulong *) (char *) psinitialsystemprocess + vadoffset );
Vadinfoarrayindex = 0;
Vadtreewalk (vadtreeroot );
Vadtreedisplay ();
}

Let's take a look at the output results of the querysystemprocessvad routine:

Vadroot is located @ fe4ddf28
VAD @ starting ending parent leftlink rrightlink flags mmci Pte
Fe4ddf28 00010000 00042fff 00000000 00000000 fe354dc8 04000000 fe4dd0e8 e1008a40
Ff874348 00060000 00060fff fe354dc8 00000000 ff8389a8 c4000001 00000000 00000000
Ff8389a8 00070000 00070fff ff874348 00000000 ff7b72c8 04000000 ff8345a8 e2747880
Ff7b72c8 00080000 0017 FFFF ff8389a8 00000000 00000000 04000000 fe33db48 e2936040
Fe354dc8 77f80000 77ff8fff fe4ddf28 ff874348 00000000 07100003 fe354b68 e1630040

Compare the result with the query command output in SoftICE with the preceding output:

: Query System
Address range flags mmci PTE name
00010000-00042000 04000000 fe4dd0e8 e1008a40
00060000-00060000 c4000001
00070000-00070000 04000000 ff8345a8 e2747880
00080000-0017f000 04000000 fe33db48 e2936040 heap
77f80000-77ff8000 07100003 fe354b68 e1630040 NTDLL. dll

Now, I have presented the two main memory mechanisms used by Windows NT/2000 user code (another way may be filemapping objects) to you, of course, the internal method involves a lot of content, and you can only rely on yourself.

References:
1. Jeffrey Richter
<Programming applications for Microsoft Windows, Fourth Edition>
2. Matt pietrek <Windows 95 system programming secrets>
3. <unauthenticated ented Windows NT> with source code