Go to: parse the working sets of Windows 2000/XP Processes

In "parsing Windows 2000/XP physical memory management", I introduced in detail the concept of page frame database, in terms of physical memory organization and management, every page system saves a structure in the page box database for tracking page status. However, the page-frame database does not really coordinate the use of physical memory. We know that Windows is a multi-task operating system, while physical memory is a relatively poor resource. To avoid a process (or system) from consuming this resource, a workingset is introduced). Workingset is a very important term for memory management. in Windows 2000/XP, it is generally divided into two types: Process Working sets and system working sets, which are used to track the physical memory usage of each process and system. Due to the introduction of Terminal Services, another kind of working set session is used to track the usage of physical memory by each session. This article briefly describes the organization and management of a working set in Windows 2000/XP Based on the Internal Organization Mode of the working set of processes.

Eprocess is the description of the process structure, so starting with eprocess, you can certainly find the expression of the process working set. In fact, the sub-structure mmsupport in eprocess is some key content about the process and the memory subsystem. The process work set is also here. For earlier kernel versions, the content is not integrated into the mmsupport structure, and the mmsupport definitions are different in versions, the definitions of mmsupport in Windows XP build 2600 sp0 are listed below (all structures in this article may only apply to this version ):

Typedef struct _ mmsupport {
Large_integer lasttrimtime;
Mmsupport_flags flags;
Ulong pagefaultcount;
Ulong peakworkingsetsize;
Ulong workingsetsize;
Ulong minimumworkingsetsize;
Ulong maximumworkingsetsize;
Pmmwsl vmworkingsetlist;
List_entry workingsetexpansionlinks;
Ulong claim;
Ulong nextestimationslot;
Ulong nextagingslot;
Ulong estimatedavailable;
Ulong growthsincelastestimate;
} Mmsupport, * pmmsupport;

In mmsupport, peakworkingsetsize, workingsetsize, minimumworkingsetsize, and maximumworkingsetsize indicate the working set peak value of the process, of course, the working set size, and the maximum and minimum allowable working sets. Both the performance generator (perfmon.msc)and Task Manager (taskmgr.exe) can trace and display these data processes. Win32 API getprocessworkingsetsize (Ex) and setprocessworkingsetsize (Ex) can obtain or set minimumworkingsetsize and maximumworkingsetsize after they have the corresponding process_query_information and process_set_quota permissions.

When a process is created, the working set of the process is always empty. CreateProcess has the responsibility to initialize the working set of the Process during process creation. It allocates a physical page and calls miinitializeworkingsetlist to initialize the working set of the process. The latter uses the created eprocess as the parameter to initialize the mmsupport structure we mentioned above. The vmworkingsetlist (mmwsl), a very important member, is defined as follows:

+ 0x000 quota: uint4b
+ 0x004 firstfree: uint4b
+ 0x008 firstdynamic: uint4b
+ 0x00c lastentry: uint4b
+ 0x010 nextslot: uint4b
+ 0x014 wsystemic: ptr32 _ mmwsystemic
+ 0x018 lastinitializedwsystemic: uint4b
+ 0x01c nondirectcount: uint4b
+ 0x020 hashtable: ptr32 _ mmwsle_hash
+ 0x024 hashtablesize: uint4b
+ 0x028 numberofcommittedpagetables: uint4b
+ 0x02c hashtablestart: ptr32 void
+ 0x030 highestpermittedhashaddress: ptr32 void
+ 0x034 numberofimagewaiters: uint4b
+ 0x038 vadbitmaphint: uint4b
+ 0x03c usedpagetableentries: [768] uint2b
+ 0x63c committedpagetables: [24] uint4b

For efficiency, Windows 2000/XP maps the structure to a fixed virtual memory address. Specified by the kernel variable mmworkingsetlist. In fact, miinitializeworkingsetlist directly references this variable to operate the vmworkingsetlist Member of the mmsupport structure. Mmworkingsetlist is located in the kernel area (0xc0503000 in Windows XP build 2600 Professional). Generally, the kernel area is shared by all processes, however, it is clear that the workingset specified by mmworkingsetlist has different mappings for each process, that is, different content, which is the same as the process page directory or page table. The latter has been tested in detail in Windows NT/2000 paging mechanism.

The workingset is used to describe the physical memory used by the process. In other words, the pages in the workingset are all in the physical memory (not replaced with pagefile. sys medium), so accessing these pages will not cause page fault. We can use virtuallock to place pages in the process worker set. On the other hand, how does the system know whether a page (using a virtual page address) exists in the work set for this process? By looking at the mmwsl definition above, we can see that Windows 2000/XP uses a hash table (hashtable) to organize these pages. Hashtable has the features of fast retrieval and is suitable for the frequent access of workingset. Another example is the Global kernel organization. For details, see analyze Windows NT/2000 kernel object organization. And windbg provide dump Global Command kernel object! The object command is the same as that provided by windbg! Wlupus is used for dump Process Working sets. For example:

Kd>! Wlupus 7

Working Set @ c0503000
Firstfree: 469 firstdynamic: 7
Lastentry 46c nextslot: 4 lastinitialized 658
Nondirect 145 hashtable: c06f4000 hashtablesize: 400

Reading the wsystemic data...
..
Virtual Address age locked referencecount
C0300203 0 1 1
C0301203 0 1 1
C0502203 0 1 1
C0503203 0 1 1
C0504203 0 1 1
C06f4203 0 1 1
C06f5203 0 1 1
C0505203 0 1 1
C0506203 0 1 1
77c47029 0 0 1
.
.
.

The wsystemic command only dumps each element of the array pointed to by the wsystemic member (mmwsystemic pointer) of vmworkingsetlist (32bit for each element ). Windbg! The virtual address column in the result obtained by the wlupus command is the content of each 32bit of wlupus. The following windbg command is used:

Kd> dd mmworkingsetlist L 1 // address of the mmwsl structure of the current process, as described earlier in this article.
805467d0 c0503000
Kd> dd c0503000 L 10 // mmwsl content
C0503000 000003b9 000003ba 00000007 000003b9
C0503010 00000004 c050450c 00000658 0000014c
--------
| _ Mmwlupus content (as defined in mmwsl above, mmwlupus is a pointer)

C0503020 c06f4000 00000400 2017001a c06f4000
| _ Hashtablesize (uint4b) hash table size
| _ Hashtable (mmwsle_hash) address (the two values will be used below)
C0503030 c0800000 00000000 0000005c 004d023a

Kd> dd c0501_c
// The result is the virtual address column (workingset
// Frequent changes. If there is a slight difference, the system may have changed ).
C050450c c0300203 c0301203 c0502203 c0503203
C05036ac c0504203 c06f4203 c06f5203 c0505203

In fact, every virtual address here is shown in the figure above. c0300203 is not just a virtual address, because workingset is in the unit of page, so 12bit of the 32bit content is used for other purposes. In fact, in Windows XP, The 32bit content is defined as mmwslentry, specifically:

Valid: POS 0, 1 bit
Lockedinws: POS 1, 1 bit
Lockedinmemory: POS 2, 1 bit
Protection: POS 3, 5 bits
Sameprotectasproto: POS 8, 1 bit
Direct: POS 9, 1 bit
Age: POS 10, 2 bits
Virtualpagenumber: POS 12, 20 bits

The wlupus command also outputs some properties of wlupus according to this low 12bit: such as age and locked. Referencecount is located in PFN. For details, see resolve Windows 2000/XP physical memory management.

The entire structure is clear now, but as mentioned above, the workingset access is very frequent, and it depends on another important member hashtable to retrieve the page with the specified virtual address in workingset. Now that we use hashtable, we will give hashfunction (if you are interested in learning how to obtain hashfunction, you can look at how miinsertwsystemic is implemented like me ).

(PVA> A) & 0x3ffffc) % (HashTableSize-1)

Here, PVA refers to the virtual address of the page, and hashtablesize refers to the size of the workingset hash table of the current process. For a given page, how can I quickly retrieve the array subscript of this page in the wsystemic array? With a hash table, the hash table is used. The description is abstract. Let's explain the problem with a specific example: We know the virtual address 77c47000 (line 77c47029 ), the tenth item not included in mmwsystemic (the array subscript is 9, that is, index is 9 ), the hashtablesize value of the working set of this process is 0x400 (this value may be changed by migrowwslehash as needed). Therefore:

(77c47029> A) & 0x3ffffc) % (0x400-1)

The value is 0x9a, so it is located in the bucket 0x9a of hashtable (starting with 0). Locate the bucket 0x9a through the hashtable address c06f4000 obtained above. What about the size of each bucket? Note that each bucket of this hashtable is defined as follows (_ mmwsle_hash ):

+ 0x000 key: uint4b
+ 0x004 index: uint4b

That is, each bucket is 8 bytes, so we can use the following KD command to get the result:

Kd> dd c06f4000 + 9A * 8 L 2
C06f44d0 77c47000 00000009

The key value is 77c47000, that is, the virtual address, and the index value is 9, which verifies the output result of the preceding windbg wsble command. Now, the workingset organization has discussed almost the same. It should be pointed out that in Windows XP, The workingset design is much more than that discussed here, for example, workingset hash tables can be expanded (via migrowwslehash), hashtable content insertion, modification, deletion, and working set trimming (via mitrimworkingset), especially working set trimming, at the beginning of the article, we mentioned that the main function of working sets is to rationally use physical memory to avoid a process (or system) Consuming physical memory. The maximum and minimum values of workingset and the quota specified by quota are used, limits the use of physical memory. If such a range exists or the physical memory is exhausted, the working set is used for trimming. Andrew Tanenbaum's modern operating systems introduces a variety of workset trimming algorithms, in a single processor, Windows 2000/XP uses algorithms more like LRU (clock algorithm is just like many UNIX systems). You should see the output age value above. Due to restrictions, I have only tried it on a single processor. For the sake of completeness, I will briefly introduce the situation of multi-processor: in Multi-processing, Windows 2000 uses the FIFO (first in first out) algorithm, but from some of the Microsoft introductions I have seen, windows XP /. NET Server 2003 also uses LRU in Multi-processing. It seems that the Windows kernel is becoming more and more perfect.

This article only introduces the process working set. It is similar to the system working set and session working set. In fact, I started writing this article only after analyzing the three working sets. These concepts and structures are constantly discovered and excited. However, I have never seen any discussions or mistakes at these structural levels, sorry, thank you!

Bytes -----------------------------------------------------------------------------------------
Article from: http://www.geocities.jp/webcrazyjp/ntws.htm (this guy used a Japanese domain name)
Author: webcrazy (http://webcrazy.yeah.net)