Windows virtual Memory mechanism

In Windows systems, each process has its own separate virtual address space. The size of this address space is related to the computer hardware, the operating system, and the application.

For 32-bit programs, you can use up to 2GB space (0X00010000-0X7FFEFFFF). In order to obtain the 3GB address space, the following methods can be expanded in different Windows systems.

1. Operating system aspects

① 32-bit windowsxp

② 32-bit Win7--Administrator privileges Execute command:bcdedit/set increaseuserva 3072 to open

③ 64-bit Win7--32-bit program is turned on by default 3GB, no additional settings required

2. Application aspects

Whether 32-bit or 64-bit Windows to enable 32-bit programs to use 3GB of memory, you must add parameters to the link: /largeaddressaware

Process address space segment

Note: Process address space at low address, operating system kernel at high address

Process address space distribution (take 2GB for example)

The Windows system specifically outlines a 0x70000000-0x80000000 (256MB) area in process space for mapping these commonly used system DLLs (such as Kernel32.dll, Ntdll.dll, etc.)

The default base address of the system DLL is adjusted to prevent collisions at load time, triggering rebasing(reset base address)

Note: The base address must be aligned to the allocation granularity (64KB)

Win7, exe in the PE file base address of 0x400000,dllprj.dll is 0x10000000 and the address is not occupied by other DLLs, but the actual exe is mapped to 0xec0000,dllprj.dll is mapped to 0x535a0000

When generating EXE and DLL modules, the link is using the parameter /dynamicbase(Enable dynamic base address)

Note: The address space layout is randomized, addressing spaces, layout randomization (ASLR): Prevent malicious programs from attacking known addresses

The Windows memory allocation process can be refined to the following 3 points:

① retains a virtual memory address space: An address space is retained from 4GB of the process. //VirtualAlloc function with Mem_reserve parameter

The start address must be an integer multiple of the system allocation granularity (64KB), and the size must be an integer multiple of the system page size (4KB).

② commits a virtual memory address space: Maps the virtual memory of the machine to a part of the address space that the process has reserved. //VirtualAlloc function with mem_commit parameter

Both the start address and the size must be an integer multiple of the page size (4KB).

③ maps the virtual memory address space to physical memory pages (RAM): When a page submitted by the access process is accessed, the physical memory page is actually allocated by a missing page break (aka page missing, page fault, pagefault) mechanism, and the address space mapping of the corresponding page is modified.

Note 1: the address that is accessed in the program must be a reserved and committed virtual memory address

NOTE 2: You can use VirtualFree to release reserved or committed virtual memory address space

Memory Metrics Concepts

Virtual Memory:

Private Bytes //process committed the number of bytes of virtual memory corresponds to the "commit size" in the Win7 Task Manager, "commit" in Resource manager

Peak Private Bytes //process committed number of bytes in virtual memory

Virtual Size //process reserved bytes

Page faults //The number of pages that have been interrupted corresponds to Win7 in the task Manager

Physical Memory:

Working Set = WS Private + ws shareable //process consumes the total number of bytes of physical memory corresponding to the "Working settings (Memory)" In the Win7 Task Manager, "Working set" in Resource manager

WS private //Process exclusive physical memory bytes (e.g. heap memory + stack memory +cow mechanism created memory) corresponds to Win7 in Task Manager "memory (dedicated working set)", "private" in Resource manager

WS shareable //processes can share physical memory bytes with other processes (e.g. EXE and DLL code snippets, data segments, etc.) corresponding to "shareable" in Win7 Resource manager

the number of physical memory bytes that WS shared//process has shared with other processes, WS Shared<=ws shareable

//If only one EXE instance is started, EXE's code snippet, data segment, etc. will not be shared, so it is not counted in WS shared

Peak Working Set //physical memory maximum number of bytes corresponds to Win7 in Task Manager (memory)

Note: Whether it is virtual memory or physical memory of the various indicators, by the statistical user state that part of the occupied

Page Interchange File

Paging file (page file): Typically used as a fallback memory for writable physical memory pages. Under Windows, the file is named Pagefile.sys, which is located in the root directory of each disk.

The size of the paging file can be set according to the hardware and software condition of the machine, and even the use of the paging file is turned off.

Page out: When there is not enough physical memory, the system frees up some of the physical memory pages that are infrequently used and has fallback, and points the virtual address mapping relationship to fallback.

① a paging file (e.g. heap, stack, etc.) as a fallback: allocate space in a paging file and copy content to it before releasing

② in memory-mapped files (such as: exe, DLL, etc.) as backup: Direct release

Page in: When a virtual memory address is read by the system, and the page that contains the address is not in the physical memory page, a fault is generated.

Tells the system to retrieve the virtual memory page that contains the address from the paging file or the memory-mapped file (that is, copy the content back to the physical memory page and create a new virtual address to map to the physical memory page, and then free up the space in the corresponding portion of the paging file).

Write-time replication mechanism

Copy-on-write mechanism (copy on Write, COW): When the Writecopy Property memory page is modified, a memory page copy is triggered to conserve physical memory and paging file usage.

Note: When mapping an EXE or DLL file, the system specifies the data page as the Page_writecopy property, and the code page is specified as the Page_execute_writecopy property

Specific process:

① When a process modifies a memory page, the system will find an idle physical memory page, copy everything to the new page, then mark the fallback memory for the new page as the paging file, and finally point the virtual memory page of the process to the new physical memory page.

② After these steps, the process can use its own copy, modify it on a new physical page, and not have any effect on the original memory page.

Reset Base Address

Reset Base Address (rebasing): When the module is loaded, if the destination address is occupied or security-based, the system assigns a new base to the module based on the size of the required address space and loads the module at the base site.

Problem:

① Once a rebasing occurs, address corrections are made to all pages in the relocation table when the module is mapped.

When the ② system corrects the pages of these addresses, the write-time replication mechanism is triggered.

address space layout randomization (Address space layout RANDOMIZATION,ASLR)

Microsoft introduced a technology called ASLR in the Vista system, which is loaded into random locations (pseudo-random) every time, preventing malicious programs from attacking known addresses.

ASLR not only randomly handles the module address, but also environment the address of the heap, stack, process environment block (process, block, PEB), Thread environment block (thread environment block, TEB).

The ASLR technology puts rebasing into the kernel for processing, which means that it is possible to minimize the occurrence of rebasing at the system-wide level (originally only in process scope), thus saving the use of physical memory and paging files.

PE file loading

Note: mappings must be in pages (4KB) and aligned by page boundaries

After the mapping is performed, most of the instructions and data are not yet loaded into physical memory. The loading process is performed dynamically as the program executes.

The process: When the CPU accesses instructions and data, it discovers that the page in which the address resides is not in the physical memory page, it triggers a page break, and the system finds an idle physical memory and loads the content from the backup (image file or paging file) into the physical memory page.

Windows virtual Memory mechanism