Understanding of the process space

(Why the process address space for a 32-bit machine is 4GB)

Before entering the topic, we will discuss the development process of the operating system memory management mechanism, so that we can understand the memory management mechanism of the current operating system better.

a early memory allocation mechanism

In an earlier computer, to run a program, all of these programs are loaded into memory, and the program is running directly in memory, which means that the memory address accessed in the program is the actual physical memory address. When your computer runs multiple programs at the same time, you must ensure that the total amount of memory used by these programs is less than the actual physical memory size of the computer. How does the operating system allocate memory for these programs when the program is running multiple programs at the same time? The following example illustrates the memory allocation method at that time:

The total memory size of a computer is 128M, now running two programs A and B, a requires memory 10M, B takes up 110 of memory. The computer takes this approach when allocating memory to the program: first allocating the first 10M in memory to program A, then dividing 110M from the remaining 118M in memory to program B. This allocation method guarantees that both program A and program B can run, but this simple memory allocation policy is a lot of problem.

Figure an early memory allocation method

Issue 1: The process address space is not isolated. Because the program is directly accessing the physical memory, the malicious program can arbitrarily modify the memory data of other processes in order to achieve the purpose of destruction. Some non-malicious, but bug-like programs may inadvertently modify the memory data of other programs, which can cause other programs to run abnormally. This situation is intolerable to the user, because one of the tasks fails, at least not affecting other tasks, when the user wants to use the computer.

Issue 2: Memory usage is inefficient. In the case of both A and B, if the user is running program C, and program C needs 20M of memory to run, while the system only has 8M of space available, so the system must be in a running program to temporarily copy the program's data to the hard disk, free up some space for program C Use, and then load the data from program C all into memory. As you can imagine, in this process, there is a lot of data in the loading and pretending, resulting in very low efficiency.

Issue 3: The address of the program run is not deterministic. When the remaining space in memory can meet the requirements of program C, the operating system randomly allocates a contiguous 20M space in the remaining space to program C, because it is randomly assigned, so the address of the program run is indeterminate.

two Segmented

In order to solve the above problems, people think of a workaround, that is, to add a middle layer, the use of an indirect address access method to access physical memory. In this way, the memory address accessed in the program is no longer the actual physical memory address, but a virtual address, which is then mapped by the operating system to the appropriate physical memory address. In this way, as long as the operating system to deal with the virtual address to the physical memory address mapping, you can ensure that the different programs eventually access the memory address in different areas, no overlap, you can achieve the effect of memory address space isolation.

4GB because one pointer length is 4 bytes in a 32-bit operating system, while the 4-byte pointer's addressing capability is from 0x00000000 ~0XFFFFFFFF, the maximum value of 0xFFFFFFFF represents a capacity of 4GB size. In contrast to the virtual address space, there is a physical address space, which corresponds to the actual physical memory. If you have 512M of memory installed on your computer, this physical address space represents a range of 0x00000000~0x1fffffff. When the operating system makes virtual addresses to physical address mappings, it can only map to this range, and the operating system will only map to that range. When a process is created, each process will have its own 4GB virtual address space. Note that this 4GB address space is "virtual", not real, and each process can only access the data in its own virtual address space, unable to access data in other processes, through this method to achieve inter-process address isolation. Is that the 4GB virtual address space application that can be used at will? Unfortunately, under the Windows system, this virtual address space is divided into 4 parts: a NULL pointer area, a user area, a 64KB forbidden zone, and a kernel zone. The application can only use the user area, about 2GB (maximum can be adjusted to 3GB). The kernel area is 2GB, the kernel zone is the system thread scheduling, memory management, device driver and other data, this part of the data for all processes to share, but the application is not directly accessible.

The reason people want to create a virtual address space is to solve the problem of process address space isolation. But to execute, the program must run in real memory, so a mapping must be established between the virtual address and the physical address. Thus, through the mapping mechanism, when a program accesses an address value on a virtual address space, it is equivalent to accessing another value in the Physical address space. People think of a segmented (sagmentation) approach, whose idea is to do a one by one mapping between the virtual address space and the physical address space. For example, a 10M size space in a virtual address space maps to a 10M size space in the physical address space. This idea is not difficult to understand, and the operating system guarantees that the address space of different processes is mapped to different regions in the physical address space, so that each process eventually accesses the

The physical address space is separate from each other. In this way, the address isolation between processes is realized. As an example, assuming that there are two processes a and B, process a requires a memory size of 10M, its virtual address space is distributed in 0x00000000 to 0x00a00000, process B requires 100M of memory, and its virtual address space is distributed from 0x00000000 to 0x06 400000. Then, according to the segmented mapping method, process A maps the area on physical memory to 0x00100000 to 0x00b00000, and process B maps the area to 0x00c00000 to 0x07000000 on physical memory. So process A and process B are mapped to different memory ranges, each of which does not overlap each other, which enables address isolation. From the application's point of view, the address space of process A is distributed across 0x00000000 to 0x00a00000, and developers only need to access the address on this interval when doing development. The application does not care what process a is mapped to the area of physical memory, so the program's running address is equivalent to saying it is OK. Figure II shows the segmented mode

Method of memory mapping.

A method of memory mapping in two segmented modes of graphs

This segmented mapping approach solves the problem one and three of the above problems, but it does not solve the problem of two, that is, the use of memory efficiency. In the segmented mapping method, each swap in and out of memory is the entire program, which can cause a lot of disk access operations, resulting in inefficiency. So this kind of mapping method is slightly coarse, particle size ratio is large. In fact, the operation of the program has a local characteristics, in a certain period of time, the program just access a small portion of the program data, that is, the program most of the data will not be used in a period of time. Based on this situation, people think of smaller memory partitioning and mapping methods, which is paging (Paging).

three page Out

The basic way to break a page is to divide the address space into many pages. The size of each page is determined by the CPU, and then the size of the page is selected by the operating system. Currently, the Inter series CPU supports 4KB or 4MB page sizes, and 4KB is currently selected on the PC. By this choice, the 4GB virtual address space can be divided into 1,048,576 pages, and 512M of physical memory can be divided into 131,072 pages. Obviously the number of pages in a virtual space is much larger than the number of pages in the physical space.

In the segmented method, the program is always loaded into memory each time the program is run, and the method of paging is different. The idea of paging is which page is used to allocate memory for which page the program runs, and the unused pages remain on the hard disk for the time being. When these pages are used, the pages are allocated memory in the physical address space, and the mappings between the pages in the virtual address space and the physical memory pages that have just been allocated are established.

The following is a description of the implementation of the paging mechanism by introducing the loading process of an executable file. An executable (PE file) is actually a collection of compiled and linked data and instructions, which is also divided into pages, and in the process of the PE file execution, the unit that it loads into memory is the page. When a PE file is executed, the operating system first creates a 4GB process virtual address space for the program. As described earlier, the virtual address space is just a middle tier, its function is to use a mapping mechanism to map the virtual address space to the physical address space, so the creation of 4GB virtual address space is not really to create space, but to create that mapping mechanism required data structure, This data structure is the page and page table.

When the data structure required for the virtual address space is created, the process begins reading the first page of the PE file. The first page of the PE file contains information such as the PE file header and Cong, and the process maps all segment one by one in the PE file to the corresponding page in the virtual address space (the length of the segments in the PE file is an integer multiple of the page lengths), based on information such as the header and the Cong. At this time the real instruction and data of the PE file has not been loaded into the memory, the operating system only based on the PE file header and other information to establish the PE file and process virtual address space in the page mapping relationship. When the CPU wants to access a virtual address used in the program, when the CPU discovers that the address does not have an associated physical address, the CPU thinks that the virtual address is on a page that is empty, and the CPU thinks it is a page fault (page Fault), and the CPU knows that the operating system has not given the PE The page allocates memory and the CPU returns control to the operating system. The operating system then assigns a page in the physical space to the PE page, maps the physical page to a virtual page in the virtual space, and then returns control back to the process, where the process restarted from the page where the error occurred just now. The page error does not occur because memory is allocated for that page of the PE file at this time. As the program executes, page faults continue to occur, and the operating system assigns the appropriate physical pages to the process to meet the needs of the process execution.

The core idea of the paging method is that when the executable file executes to page x, it assigns a memory page y to page x, and then adds the memory page to the mapping table of the process virtual address space, which is equivalent to a y=f (x) function. The application is able to access the Y page associated with the X page through this mapping table.

Summarize:

+ bit of The address space of the CPU is 4GB, so the maximum value of virtual memory is 4GB, and the Windows operating system divides this 4GB into 2 parts, that is, 2G user space and 2G system space, system space is shared by each process, he is storing the operating system and some kernel objects, etc., while the user empty is allocated to each process, user space includes: program code and data, heap, shared library, stack.

Reference

Understanding of the process space