Windows Memory Management Mechanism and C ++ memory allocation instance (I): process space

Background:

In programming, many windows or C ++ memory functions do not know the difference, let alone effective use. The root cause is that they do not have a clear understanding of the memory management mechanism of the operating system, this article attempts to clarify this mechanism through a summary description and examples.

Purpose:

Understand the Windows Memory Management Mechanism, and effectively use C ++ memory functions to manage and use the memory.

Content:

This article consists of six sections, which are published by section due to the large space.

1. process address space

1.1 address space

· 32 | 64-bit system | CPU

The operating system runs on the hardware CPU, the 32-bit operating system runs on the 32-bit CPU, and the 64-bit operating system runs on the 64-bit CPU. Currently, there is no real 64-bit CPU.

32-bit CPUs can only operate on 32-bit binary data at a time. The more complicated the multi-bit CPU design is, the simpler the software design is.

The software process runs on a 32-bit system, and its addressing bit is also 32-bit. It can represent a space of 232 = 4G, ranging from 0x0000 0000 ~ 0 xFFFF FFFF.

· NULL pointer Partition

Range: 0x0000 0000 ~ 0x0000 FFFF

Purpose: protect illegal Memory Access

Example: If the memory allocation fails for some reason, a null pointer 0x0000 0000 is returned, the system will exit due to access violation.

So why do we need such a large area? isn't an address value enough? I was wondering, isn't it because we don't allow 8 or 16-bit programs to run on 32-bit systems ?! Because the null partition has a process space of 16.

· Exclusive user Partition

Range: 0x0001 0000 ~ 0x7ffe FFFF

Purpose: A process can only read or access virtual addresses in this range. Any behavior beyond this range will result in illegal exit.

Example:

Most of the addresses used in the program's binary code are in this range, and all the EXE and DLL files are loaded to this. Each process has nearly 2 GB of space.

NOTE: If/3G is set on boot. ini, the range of this region is increased from 2g to 3G: 0x0001 0000 ~ 0 xbffe FFFF.

· Sharing kernel partitions

Range: 0x8000 0000 ~ 0 xFFFF FFFF

Purpose: This space is used for operating system kernel code, device drivers, device I/O cache, non-page memory pool allocation, Process Table and page table.

Example:

Each process of this address can be shared.

NOTE: If/3G is set on boot. ini, the range of this region is reduced from 2g to 1G: 0xc000 0000 ~ 0 xFFFF FFFF.

Through the above analysis, we can know that if the system has N processes, the required virtual space is: 2G * n + 2G (the kernel only needs 2 GB of shared space ).

 

1.2 address ing

· Region

A region refers to a continuous address in the preceding address space. The area size must be an integer multiple of the granularity (64 K). If it is not, the system automatically processes it as an integer multiple. Different CPU granularities vary, most of which are 64 KB.

The status of a region includes idle, Private, ing, and image.

In your application, the space application process is called reservation. You can use virtualalloc. The space deletion process is release and virtualfree.

After the address space is reserved in the program, you cannot access the data, because you have not paid yet, and there is no real Ram associated with it.

At this time, the region status is private;

By default, the region status is idle;

When the EXE or DLL file is mapped into the process space, the region status changes to the image;

When a data file is mapped to a process space, the region status changes to ing.

· Physical storage

Windows series support different memory ceilings, from 2 GB to 64 GB. Theoretically, the 32-bit CPU can only support addressing 4G memory on the hardware, and the memory exceeding 4G can only be compensated by other technologies. By the way, Windows Personal Edition only supports a maximum of 2 GB of memory. Intel uses address windows extension (AWE) technology to make the addressing range 236 = 64 GB. Of course, the operating system must also work together.

The minimum unit of memory allocation is 4 K or 8 K. Generally, the Unit varies depending on the CPU. You can see that the system function can be used to obtain the region granularity and page granularity.

· Page files

A page file is a system file on a hard disk. Its size can be set in the system attribute. It is equivalent to physical memory, so it is called virtual memory. In fact, its size is the key to affecting the system speed, if the physical memory is not large.

The size of each page is the same as the minimum unit for memory allocation, usually 4 K or 8 K.

· Access attributes

The access attribute of a physical page refers to the specific operations on the page: readable, writable, and executable. The CPU generally does not support executable operations. It considers readable as executable. However, the operating system provides the executable permission.

Page_noaccess

Page_readonly

Page_readwrite

Page_execute

Page_execute_read

Page_execute_readwrite

These six attributes are well understood. The first one is to reject all operations, and the last one is to accept received operations;

Page_writecopy

Page_execute_writecopy

These two attributes are very useful when running multiple instances of the same program; they allow the program to share the code segment and data segment. Generally, multiple processes read-only or execute pages. If you want to write them, the page will be copied to the new page. These two attributes can be set when the EXE file is mapped.

Page_nocache

Page_writecombine

These two are required when developing device drivers.

Page_guard

When a byte is written to the page, the application receives a stack overflow notification, which is useful in the thread stack.

· Ing Process

The address of the process address space is a virtual address, that is, when a command is obtained, the virtual address must be converted to a physical address to access data. This operation is performed by the Page Object and page table.

It can be seen that the page size is 4 K, and each item (32-bit) stores the physical address of a page table; each page table size is 4 K, and each item (32-bit) stores the physical address of a physical page. There are a total of 1024 page tables. The 4 K + 4 K * 1 k = 4.4m space can represent the process's 1024*1024*(4 K per page) = 4G address space.

The 32-bit address in the process space is as follows:

 

The top 10 digits are used to locate one of the 1024 page items. After the physical address of the page table is retrieved, the value of the page table item is obtained using the middle 10 digits. Based on the address worth accessing the physical page, because a page has 4 K size, we can get the unit address with 12 low bits so that we can access this memory unit.

Each process has its own page view and page table. How does the initial process find the physical page where the page view is located? The answer is that the CPU's Cr 3 register will save the physical address of the current process page.

When a process is created, both the page view and page table need to be created, which requires a total of 4.4 MB. In the process space, 0xc030 0000 ~ 0xc030 0fff is used to save 4 K space for the page. 0xc000 0000 ~ 0xc03f FFFF is the 4 m space used to save the page table. That is to say, you can access these addresses in the program to read the specific values of the Page Object and page table (in kernel mode ). What I don't understand is that the page table space contains the page destination space!

As for whether the page view and page table are stored in physical memory or page files, I think the page view is common and should be more likely in physical memory, when the page table needs to be imported from the page file to the physical memory.

The page item and page table item are a 32-bit value. When the 0th-bit value of the page item is 1, the page table is already in the physical memory. When the 0th-bit value of the page table item is 1, indicates that the accessed data is already in the memory. There are also many indicators such as whether the data has been changed and whether the data can be read and written. In addition, when the 7th-bit value of the page item is 1, it indicates that this is a 4 m page. This value is already the physical page address, and the 22-bit lower of the virtual address is used as the offset. There are also many other indicators: whether the data has been changed, whether it can be read and written, and so on.

 

1.3 example

· Compile and generate the software program exe

The software description is as follows:

Main ()

{

1: define global variables

2: process function logic (dll library required for load, call method processing logic)

3: define and implement various methods (methods contain local variables)

4: program ended

}

Compile the program to generate the EXE file with the required DLL library.

· EXE file format

The EXE file has its own format and has several sections:. Text is used to put binary code (exe or DLL);. Data is used to put various global data.

. Text

Command 1: Move a, B

Command 2: Add a, B

...

. Data

Data 1: A = 2

Data 2: B = 1

...

These addresses are virtual addresses, that is, the address space of the process.

· Run the EXE program

Process Creation: When this exe program is run, the system will create a process, establish a process control block PCB, generate the process page and page table, and put it in the PCB.

 

Data Alignment: the data memory address is divided by the data size. If the remainder is 0, the data is alignment. The current compiler considers data alignment during compilation. After an EXE file is generated, the data is basically alignment. When the CPU is running, the Register indicates whether the CPU can automatically align the data, if alignment is not possible, you can access the memory twice or notify the operating system to handle the issue.

Note that if the data is not aligned, the CPU processing efficiency is very low.

 

File ing: the system will not load the entire EXE file and all DLL files into the physical memory, nor will it be loaded into the page file. On the contrary, it establishes a file ing, that is, using the EXE itself as a page file. The system loads some binary code into the memory and assigns the page to it.

Assume that a page is assigned with a physical address of 0x0232 fff1. The virtual address of a mounted command is 0X4000 1001 = 0100 0000 00 0000 0000 01 0000 0000 0001. If a page has 4 kb, the system will save the command at the address with a low 12-bit 0x0001. At the same time, the system finds the page items based on the 10-bit high 0x0100. If there is no associated page table, the system generates a page table and assigns a physical page. Then, locate the table item based on the 10-bit 0x0001 and store the physical address 0x0232 fff1.

 

Execution Process:

During execution, when the system obtains a virtual address, it finds the data address based on the page and page table. Based on the value on the page, it can determine whether the page table is in the page file or in the memory;

If it is in a page file, the page will be imported into the memory to update the page items. After reading the value of the page table item, you can determine whether the data page file is still in the physical memory; if it is in the page file, it will be imported to the memory to update the page table item. Finally, the data is obtained.

During the process of allocating physical pages, the system will appropriately remove temporarily unused pages based on the memory allocation situation. If the page content changes (by the flag of the page table item ), save to the page file. The system maintains the correspondence between the memory and the page file.

Because the EXE file is used as a memory ing file, when you need to change the data, such as changing the global variable value, use the copy-on-write mechanism to regenerate the page file, save the result to this page file. The original page file still needs to be used by other process instances.

After knowing how commands and data are imported into the memory and how to find them, the rest is the process in which the CPU continuously retrieves commands, runs, and saves data. When the process ends, the system clears the previous structures, releases related physical memory, and deletes page files.