Explore Windows 2000/XP prototype PTE (http://webcrazy.yeah.net)

Explore the Windows 2000/XP prototype Pte
WebSphere (http://webcrazy.yeah.net)

Memory Management is the most important and complex process of operating system implementation. Memory sharing has become an important and effective method for relatively poor memory resources. The implementation of Windows 2000/XP in this respect relies on a software mechanism called prototype PTE (PPTE. In Windows NT/2000 paging mechanism, I introduced in detail how intel X86 implements segmentation and paging hardware Pte. Let's review this mechanism:

Assume that one of our processes maps the 4 m space starting with the virtual address 0 XXXXXXXXX (assuming that it is located at the allocation granularity, the current 4 m space is mapped to the actual physical memory (which is rare in Windows 2000/XP due to lazy evaluation and other advanced ideas ). We divide the 4 m space into 4 K (page_size, determined by the x86 processor) of 1000 blocks. For the nth 4 K (0 <= n <1000 ), its Virtual Address (0 XXXXXXXXX + N * 4 K), we have a corresponding hardware PTE, pointing out that the current 4 K reside in the physical memory location. The hardware PTE can be located through the pdbr (Cr 3 register) and virtual address (for details, see Windows NT/2000 paging mechanism).

Now let's consider this situation. We have a file with a size of 4 MB. We know that we usually need to read this file into the memory. Imagine there are two or more processes that need to read and write the file at the same time. This requires Memory Sharing. In fact, even if only one process accesses this file, Windows 2000/XP will consider the sharing situation in advance for such files that may need to be shared. She achieved this goal through a kernel object called Section. In this case, Memory sharing is not only the full use of memory resources, even if we can allocate 4 MB space for each process, but this will lead to various processes may not get the latest content of this file at some time. This is a very bad situation. In Windows 2000/XP, the prototype Pte is used to solve this problem. Based on the same principle of hardware PTE, for such a 4 m file, when ing this file, Windows 2000/XP also divides this file into 1000 blocks, each of which is 4 K (page_size) size. Then, 1000 Dwords are allocated from the Page Swap area. Each DWORD value is a prototype PTE, which forms a prototype PTE table. For the nth 4 K (0 <= n <1000) of this file, if it is currently resident in the physical memory, its corresponding PPTE valid bit (bit 0, the same as the hardware PTE) is 1, and the PPTE's page frame number (20-bit higher than the PPTE) is used to indicate the physical memory. If it is still in the disk, the valid is 0. In this case, we use the high 20-bit (PFN entry) of the PPTE to find the page frame datbase (located by the matrix database), and use the subsection PTE (repte in windbg) of the PFN entry, inside Windows 2000 is called original PTE, and Windows XP is called subsection PTE). Locate the subsection, and then use the file_object of the control area pointed to by the subsection to offset N from the PPTE table in the PPTE table, formula:

PFN entry subsection PTE-> subsection-> Control Area-> fileobject + N * 4 K

Locate the file offset to be accessed, so that Windows 2000/XP reads the content from this page by calling Io on the page, and updates the ppte of the ppte table. The above positioning conversion algorithms, such as how subsection PTE locates subsection, will be introduced separately. The above description solves a very important problem. We do not need to update the hardware PTE of all processes that reference this page, because the PTE of all processes points to PPTE at this time, we only need to update PPTE to achieve our goal. As for how the process PTE points to PPTE, I will discuss this content below. Here you only need to have a concept. In order to point to PPTE, the process's PTE must be an invalid PTE, that is, bit 0 is 0, and its bit 10 is 1 (PPTE mark, for details, refer to the hardware_pte_x86 structure listed in "explore Windows NT/2000 copy on write mechanism ).

For PPTE, because the x86 processor does not provide such a method, like processing the hardware PTE, the CPU directly performs address conversion. When processing page fault, the Windows 2000/XP Memory Manager simulates this implementation through software mechanisms. This is an essential difference between hardware PTE and PPTE.

It should be emphasized that PPTE exists in the Page Swap area (the position specified by the matrix endpoint and the matrix endpoint starts from the virtual address 0xe000000), and may also be page out, windows 2000/XP uses micheckprotoptepagestate to determine whether the page is out, and the starting address 0xe0000000 In the Page Swap area is used to convert the invalid Pte to the address of the prototype Pte. I will introduce it later.

In the example, we use SoftICE to verify our previous description:

: Bpint E

As long as we intercept this hardware interruption, we will know that page fault has occurred, but we are not sure that this is caused by the invalid PTE pointing to the PPTE. In fact, page fault occurs in other mechanisms such as copy on write (exploring the Windows NT/2000 copy on write mechanism is discussed in detail ). However, as we mentioned earlier, PPTE's bit 10 is 1, we can easily determine whether a page fault is caused by an invalid PTE pointing to PPTE. Since the virtual address where page fault occurs is specified by the Cr2 register, after several attempts, we will continue to discuss the following:

Break due to bpint 0e (ET = 2.23 seconds)
: CPU

Processor 00 registers
----------------------
CS: EIP = 0008: 801648a4 SS: ESP = 0010: fcbeadc8
Eax = c002100b EBX = 77e74a02 ECx = 00000102 edX = 00000000
ESI = 00085108 EDI = 000493e0 EBP = 0140ff74 EFL = 00000006
DS = 0023 es = 0023 FS = 0038 GS = 0000

Cr0 = 8000003b pe mp ts et ne PG
Cr2 = 77d3bb26 // The virtual address where page fault occurs.
.
.
.

: Page 77d3bb26
Linear Physical attributes
77d3bb26 NP 01a714f6

From the point 10 of the PTE value 01a714f6 is 1, we know this is an invalid PTE pointing to the PPTE. Through the query command, we can find the address specified by CR2, which is located in module rpcrt4.dll. We can see from the following:

: Query 77d30000
Context address range flags mmci PTE name
Explorer 77d20000-77d8e000 07100001 ff8d1328 e169c580 rpcrt4.dll

Based on the introduction in my article, we use the following calculations:

:? (77d3bb26-77d20000)/1000*4 + e169c580
Unsigned long = 0xe169c5ec,-513161748, "/xe1i/xc5/xec"

We can see that PTE 01a714f6 should point to 0xe169c5ec. At this time, the control area directed by mmci can be read from the formula I mentioned above, that is, the rpcrt4.dll offset (0xe169c5ec-0xe169c580)/4*1000, that is, 4 K at 0x1b000, read the virtual address 77d3b000 (0xe169c5ec-0xe169c580)/4*1000 + 77d20000), and the address 77d3bb26 specified by Cr2 must be in this 4 K.

In fact, we have already described a typical process of mmaccessfault processing invalid PTE pointing to PPTE. Here we only demonstrate that the page pointed to by the prototype PTE does not reside in the physical memory. Imagine if our page is already in the physical memory, do we still need to find VAD in time? This involves how to locate the prototype PTE for an invalid PTE, so I have been using the invalid PTE command pointing to the PPTE. Inside Windows 2000 states the specific format of the invalid PTE pointing to the PPTE, but I find that the description is not correct, I have always believed that there will be no problem for anyone who can reach the Windows 2000 code like the author, so I am stuck here when I understand PPTE. Later, we realized through disassembly and found that we calculated the position of the PPTE using the following method:

(PTE> 2) & 0x3ffffe00 + (PTE & 0x000000ff) <1 + 0xe000000

Here, Pte is the invalid PTE pointing to PPTE, and 0xe0000000 is the starting address of the Page Swap area. We also use the above example to demonstrate this algorithm:

The above invalid Pte is 01a714f6. With this value, we can get:

PPTE address = (0x01a714f6> 2) & 0x3ffffe00 + (0x01a714f6 & 0x000000ff) <1 + 0xe000000
= 0x0069c53d & 0x3ffffe00 + 0xf6 <1 + 0xe000000
= 0x69c400 + 0x1ec + 0xe000000
= 0xe169c5ec

It is consistent with the PPTE position 0xe169c5ec found through VAD.

To better understand PPTE, let's look at an example. We know that NTDLL. dll is a very important DLL in Windows 2000/XP. As long as the operating system starts normally, NTDLL will be shared by multiple processes. We use SoftICE for the following analysis:

: Query-x 77f50000
Context address range flags mmci PTE name
SMSs 77f50000-77ff8000 07100005 80e6fa50 e0000f9e8 NTDLL. dll
.
.
.
Explorer 77f50000-77ff8000 07100005 80e6fa50 e0000f9e8 NTDLL. dll
.
.
.

: ADDR SMSs
: Mod NTDLL
Hmod base peheader Module name file name
77f50000 77f500e8 NTDLL/Windows/system32/NTDLL. dll

Based on the base address of NTDLL 77f50000, we can view its hardware PTE:

: Dd 1df * 1000 + 350*4 + c0000000 L 4 // For details, refer to Windows NT/2000 paging mechanism.
0010: c01dfd40 02267027 02f2e005 02f2f005 00c7e4fa 'P &.............

From these page tables of the SMSs process, we can easily understand NTDLL. DLL 1st to 3 4 K reside in the physical memory address, because they are both valid hardware PTE, and the fourth PTE (00c7e4fa ), although it is an invalid PTE (bit 0 is 0), because it is a PTE (bit 10 is 1) pointing to PPTE ), therefore, we cannot determine NTDLL only because this Pte is an invalid Pte. the 4th k dll is not in the physical memory. We need to further analyze this PTE and find the PPTE pointing to determine whether the 4th K is actually on the disk. OK. Through the algorithm mentioned above, we can easily calculate that the PPTE address is e0000f9f4. Let's take a look at the value of this PPTE:

: Dd e0000f9f4 L 4
0010: e0000f9f4 02f30121 02f31121 02f32121 02f33121 !...!...!!..! 1 ..

From the value 02f30121, we can determine that the 4th 4 K also exists in the physical address, located in the physical memory with page frame number 02f30, and the rest is to query the PFN database.

We can also view the ntdll. dll ing of the explorer process to verify this situation:

: ADDR Explorer
: Dd 1df * 1000 + 350*4 + c0000000 L 4
0010: c01dfd40 02267025 02f2e025 02f2f025 02f30025% P &. % ..%...

This is clear. I mentioned at the beginning of this article: "We do not need to update the hardware PTE of all processes that reference this page, because the PTE of all processes points to PPTE at this time, we only need to update PPTE to achieve the goal ". We can also see that the 4th 4 K of NTDLL. dll is actually in the physical memory, but Windows 2000/XP does not update each PTE that references this page, just like the SMSs process. However, PPTE has already pointed to its actual address. When the SMSs process accesses this region for the first time, the memory manager will only 02f30025 (assuming that the attributes are the same as those of the explorer process using this page and are considered as the access Bit Flag) this effective hardware PTE updates the 00c7e4fa above. Now everything is clear.

Although this article focuses on PPTE, I have made it very clear about the internal mechanism of the section object. This is what I used to set the title of an article as an Analysis Section. My understanding of PPTE has gone through a lot of time, mainly because currently, this part of information is really unavailable, the only "Inside Windows 2000" that points to PPTE's invalid PTE format is not explicitly described (especially with 0xe000000, which makes me suffer ), the format described in this article has been tested on Windows 2000 and XP. In fact, one of the two examples in this article is demonstrated on Windows 2000 Server build 2195, and the other on XP Professional build 2600.

After this introduction of PPTE, let's review several links in the Memory Manager:

In fileobject, section_object_pointers-> datasectionobject or section_object_pointers-> imagesectionobject (determined by the file opening method mapped to the section object) points to control area, at the same time, the mmci member (called by SoftICE) of the VAD that describes the virtual address mapped to this file also points to this control area. There are at most subsections under the control area, and the subsection points to PPTE, the PPTE table is usually located at the bottom of the segment structure pointed to by control area. The section object points to the segment; the process page table points to the PPTE; all this is clearly described. There is another major link, that is, the restore PTE (original PTE) of the PFN entry points to subsection. I will introduce this link next time.

From Windows NT/2000 paging mechanism to PPTE introduction today, I have a deep understanding of Windows 2000/XP memory management, as for the unmentioned concepts such as working set, it is also very important. Having gone through a lot of obfuscation, the memory manager has some concepts. All the discussions are based on your own understanding, please give me more advice on the right and wrong (tsu00@263.net ).