Talking about process address space and virtual storage space

early memory allocation mechanism

in an earlier computer, to run a program, all of these programs are loaded into memory, and the program is running directly in memory, which means the memory addresses that are accessed in the program are the actual physical memory addresses . When your computer runs multiple programs at the same time, you must ensure that the total amount of memory used by these programs is less than the actual physical memory size of the computer.

How does the operating system allocate memory for these programs when the program is running multiple programs at the same time? The following example illustrates the memory allocation method at that time:

The total memory size of a computer is 128M, now running two programs A and B, a requires memory 10M, B takes up 110 of memory. The computer takes this approach when allocating memory to the program: first allocating the first 10M in memory to program A, then dividing 110M from the remaining 118M in memory to program B. This allocation method guarantees that both program A and program B can run, but this simple memory allocation policy is a lot of problem.

Early memory allocation methods

Question 1 : The process address space is not isolated. Because the program is directly accessing the physical memory, the malicious program can arbitrarily modify the memory data of other processes in order to achieve the purpose of destruction. Some non-malicious, but bug-like programs may inadvertently modify the memory data of other programs, which can cause other programs to run abnormally. This situation is intolerable to the user, because one of the tasks fails, at least not affecting other tasks, when the user wants to use the computer.

Question 2 : Inefficient memory usage. In the case of both A and B, if the user is running program C, and program C needs 20M of memory to run, while the system only has 8M of space available, so the system must be in a running program to temporarily copy the program's data to the hard disk, free up some space for program C Use, and then load the data from program C all into memory. As you can imagine, in this process, there is a lot of data in the loading and pretending, resulting in very low efficiency.

Question 3 : The address of the program run is not deterministic. When the remaining space in memory can meet the requirements of program C, the operating system randomly allocates a contiguous 20M space in the remaining space to program C, because it is randomly assigned, so the address of the program run is indeterminate.

Segmented

in order to solve the above problems, people think of a workaround, that is, to add a middle layer, the use of an indirect address access method to access physical memory. In this way, the memory address accessed in the program is no longer the actual physical memory address, but a virtual address, which is then mapped by the operating system to the appropriate physical memory address . In this way, as long as the operating system to deal with the virtual address to the physical memory address mapping, you can ensure that the different programs eventually access the memory address in different areas, no overlap, you can achieve the effect of memory address space isolation.

When a process is created, the operating system assigns the process a  4GB  The size of virtual process address space is that the virtual address space application that  4GB  is free to use? Unfortunately, under the  Windows  system, this virtual address space is divided into  4  part: null  pointer area, user area,  64KB  Forbidden Zone, kernel area.

1)NULL pointer area (0X00000000~0X0000FFFF): If a thread in the process attempts to manipulate data in this partition, the CPU will raise illegal access. His role was to call a memory allocation function such as malloc, which would return NULL if it could not find enough memory space. Without a security check. It simply assumes that the address is assigned successfully and begins accessing memory address 0x00000000 (NULL). Because access to this partition of memory is forbidden, illegal access occurs and the process is terminated.

2) user mode partition (0X00010000~0XBFFEFFFF): This partition holds the private address space of the process. One process cannot access the data in this partition in any way (the same exe, through Copy-on-write to complete the address isolation). (in Windows, all. exe and dynamic-link libraries are loaded into this zone.) The system maps all the memory-mapped files that the process can access to this partition.

2) Containment Area (0XBFFF0000~0XBFFFFFFF): This partition is forbidden to enter. Any attempt to access this memory partition is illegal. The purpose of Microsoft's retention of this partition is to simplify the reality of the operating system.

3) kernel area (0XC0000000~0XFFFFFFFF): This partition holds the code that the operating system resides on. Thread scheduling, memory management, file system support, network support, and all device driver code are loaded on this partition. This partition is shared by all processes.

The application can only use the user area, about 2GB (maximum can be adjusted to 3GB). The kernel area is 2GB, the kernel zone is the system thread scheduling, memory management, device driver and other data, this part of the data for all processes to share, but the application is not directly accessible.

The reason people want to create a virtual address space is to solve the problem of process address space isolation . But to execute, the program must run in real memory, so a mapping must be established between the virtual address and the physical address. Thus, through the mapping mechanism, when a program accesses an address value on a virtual address space, it is equivalent to accessing another value in the Physical address space . People think of a segmented (sagmentation) approach, whose idea is to do a one by one mapping between the virtual address space and the physical address space. For example , a 10M size space in a virtual address space maps to a 10M size space in the physical address space. This idea is not difficult to understand, and the operating system guarantees that the address space of different processes is mapped to different regions in the physical address space, so that each process is eventually accessed.

The physical address space is separate from each other. In this way, the address isolation between processes is realized. As an example , assuming that there are two processes a and B, process a requires a memory size of 10M, its virtual address space is distributed in 0x00000000 to 0x00a00000, process B requires 100M of memory, and its virtual address space is distributed as 0x000000 00 to 0x06400000. Then, according to the segmented mapping method, process A maps the area on physical memory to 0x00100000 to 0x00b00000, and process B maps the area to 0x00c00000 to 0x07000000 on physical memory. So process A and process B are mapped to different memory ranges, each of which does not overlap each other, which enables address isolation. From the application's point of view, the address space of process A is distributed across 0x00000000 to 0x00a00000, and developers only need to access the address on this interval when doing development. The application does not care what process a is mapped to the area of physical memory, so the program's running address is equivalent to saying it is OK. Shows the memory-mapping method in segmented mode:

A method of memory mapping in segmented mode

This segmented mapping approach solves the problem one and three of the above problems, but it does not solve the problem of two, that is, the use of memory efficiency. In the segmented mapping method, each swap in and out of memory is the entire program, which can cause a lot of disk access operations, resulting in inefficiency. So this kind of mapping method is slightly coarse, particle size ratio is large. In fact, the operation of the program has a local characteristics, in a certain period of time, the program just access a small portion of the program data, that is, the program most of the data will not be used in a period of time. Based on this situation, people think of smaller memory partitioning and mapping methods, which is paging (Paging).

page Out

The basic way to break a page is to divide the address space into many pages. The size of each page is determined by the CPU, and then the size of the page is selected by the operating system. Currently, the Inter series CPU supports 4KB or 4MB page sizes, and 4KB is currently selected on the PC. By this choice, the 4GB virtual address space can be divided into 1048576 pages, and 512M of physical memory can be divided into 131,072 pages. Obviously the number of pages in a virtual space is much larger than the number of pages in the physical space .

In the segmented method, the program is always loaded into memory each time the program is run, and the method of paging is different. The idea of paging is which page is used to allocate memory for which page the program runs, and the unused pages remain on the hard disk for the time being. When these pages are used, the pages are allocated memory in the physical address space, and the mappings between the pages in the virtual address space and the physical memory pages that have just been allocated are established.

The following is a description of the implementation of the paging mechanism by introducing the loading process of an executable file. An executable (PE file) is actually a collection of compiled and linked data and instructions, which is also divided into pages, and in the process of the PE file execution, the unit that it loads into memory is the page. When a PE file is executed, the operating system first creates a 4GB process virtual address space for the program. As described earlier, the virtual address space is just a middle tier, its function is to use a mapping mechanism to map the virtual address space to the physical address space, so the creation of 4GB virtual address space is not really to create space, but to create that mapping mechanism required data structure, This data structure is the page and page table.

When the data structure required for the virtual address space is created, the process begins reading the first page of the PE file. The first page of the PE file contains information such as the PE file header and Cong, and the process maps all segment one by one in the PE file to the corresponding page in the virtual address space (the length of the segments in the PE file is an integer multiple of the page lengths), based on information such as the header and the Cong. At this time the real instruction and data of the PE file has not been loaded into memory, the operating system only according to the PE file header and other information to establish the PE file and process virtual address space in the mapping of the page. When the CPU wants to access a virtual address used in the program, when the CPU discovers that the address does not have an associated physical address, the CPU thinks that the virtual address is on a page that is empty, and the CPU thinks it is a page fault (page Fault), and the CPU knows that the operating system has not given the PE The page allocates memory and the CPU returns control to the operating system. The operating system then assigns a page in the physical space to the PE page, maps the physical page to a virtual page in the virtual space, and then returns control back to the process, where the process restarted from the page where the error occurred just now. The page error does not occur because memory is allocated for that page of the PE file at this time. As the program executes, page faults continue to occur, and the operating system assigns the appropriate physical pages to the process to meet the needs of the process execution.

The core idea of the paging method is that when the executable file executes to page x, it assigns a memory page y to page x, and then adds the memory page to the mapping table of the process virtual address space, which is equivalent to a y=f (x) function. The application is able to access the Y page associated with the X page through this mapping table.

the difference between a logical address, a linear address, a physical address, and a virtual address

logical addresses (Logical address) refers to the part of the offset address that is generated by the program and related to the segment. For example, if you are programming C, you can read the pointer variable itself (& operation), which is actually the logical address, which is relative to the address of your current process data segment and is not coherent with an absolute physical address. Only in Intel Real mode will the logical address be equal to the physical address (because there is no fragmentation or paging mechanism in real mode, the CPU does not perform automatic address translation), and the logic is the offset address of the code snippet in Intel protected mode (assuming that the code snippet, the data segment is identical). The application programmer only has to deal with logical addresses, and the staging and paging mechanisms are completely transparent to you and are only covered by system programmers. While an application can manipulate memory directly, it can only operate on the memory segments assigned to you by the operating system.

linear addresses (Linear address) is the middle tier between the logical address and the physical address transformation. The code generates a logical address, or an offset address in a segment, with the base address of the corresponding segment generating a linear address. If the paging mechanism is enabled, the linear address can then be transformed to produce a physical address. If the paging mechanism is not enabled, then the linear address is directly the physical address. The Intel 80386 has a linear address space capacity of 4G (2 of 32 address bus addressing).

Physical addresses (physical address) is the address signal of addressing physical memory on the external address bus of the current CPU, which is the final result address of the address transformation. If the paging mechanism is enabled, the linear address is transformed into a physical address using the items in the page directory and the page table. If the paging mechanism is not enabled, then the linear address becomes the physical address directly.

virtual memory A very appropriate analogy is : You don't have to take a long track to get a train from Shanghai to Beijing. You only need long enough rails (say 3 km) to complete this task. The way to do this is to put the rear rails immediately in front of the train, as long as your operation is fast enough to meet your needs, and the train can run like a complete track. This is the task that virtual memory management needs to accomplish. In the Linux0.11 kernel, each program (process) is divided into a total capacity of 64MB of virtual memory space. The logical address range of the program is therefore 0x0000000 to 0x4000000. Sometimes we also refer to logical addresses as virtual addresses and virtual memory space is similar, the logical address is also independent of the actual physical memory capacity. The "gap" between the logical address and the physical address is 0xC0000000, due to the exact difference between the virtual address, the linear address, and the physical address mapping. This value is specified by the operating system. The mechanism logical address (or virtual address) to the linear address is automatically converted by the CPU's segment mechanism. If paging management is not turned on, the linear address is the physical address. If paging management is turned on, then the system program needs to convert the parameters and the linear address to the physical address. This is done by setting the page Catalog table and page table entries.

Turn from: Swim hand good string stroll graffiti

Talking about process address space and virtual storage space