[CSAPP9] Virtual Storage/virtual Memory/VM Learning notes

For each process, it requires a certain size of (physical) memory (DRAM) at run time, and one of the simplest methods is how much memory the process needs to "preempt" the large (physical) memory. To give a simple example, assume that the memory size of a computer is 128MB, process A, B, C run time requires 100MB, 10MB, 30MB of memory, if according to a, B, C run order, process a runtime preemption 100MB memory, process B needs 10MB, then process A, B can run at the same time, when process C needs to run, because the system memory space is only 18MB left, not enough process C to run, then the system needs to terminate process A and release process A's memory for process C to run, its simplified icon is as follows.

Here are a few things we can see from the image above

(1) The memory between different processes is not necessarily continuous;

(2) If process C is a malicious process, it can access (for example, pointers) process B in some way and destroy it.

There are several drawbacks to this kind of memory management, first, in the case of memory constraints, if there is a large memory-intensive process, you need to make large-scale memory release so that the process can run, and secondly, in physical memory, the different processes are "visible", that a process can be "access" in some way Other processes, which creates a great deal of insecurity.

Virtual memory/Virtual storage can be said to solve the above problems, and become an important mechanism of operating system memory management. Compared to the direct "preemption" of physical memory, it provides a large, contiguous, and private, "virtual" storage space-virtual memory-on disk for each process. At the same time, this virtual memory is divided into "small pieces", when running to this part of the time to swap it to physical memory, the specific principle will be discussed later.

Here is a small summary: virtual memory (VM) can be said to be an important mechanism of operating system memory management, providing a large, consistent and private address space for the process, providing several important capabilities:

(1) It regards main memory (DRAM) as the cache of the address space stored on disk;

(2) Provide a consistent address space for each process;

(3) Protection of each address space is not destroyed by other processes.

Said so much, in fact, the most important is such a process. When we create a process, the operating system will allocate a "virtual" space for us on disk space, we will name this space as virtual memory/virtual memory, but the disk read and write speed is very slow (relative to physical memory), so when we need a part of the virtual memory data, The operating system loads this part into physical memory for execution. Since this mapping/loading process is performed automatically by the operating system, there is no "feel" or impact during process execution. Typically, for a 32-bit operating system, 4GB of virtual memory is allocated (4GB virtual memory is divided into 1GB of kernel space for Linux, 3GB of user space), and 64-bit operating systems can allocate 256TB of virtual memory (in fact, for 64-bit systems, At this stage generally support to 48-bit addressing, so the maximum support 256TB).

Next, we'll focus on how the operating system maps virtual memory to physical memory.

One, virtual memory and physical memory mapping

The virtual memory that the operating system allocates for the process can be seen as a large array, and the allocated virtual memory can be larger than the actual physical memory. For ease of operation, the operating system actually divides virtual memory into chunks for transmission, which we call virtual pages (PAGE,VP), the size of P bytes (P=pow (2,p)), and, similarly, the partitioning of physical memory into physical pages (physical page,pp). Its size is also P-byte (we sometimes call it a physical page frame).

In the case of virtual memory, there are generally two situations, unallocated space and allocated space, where the allocated space can be divided into cached and non-cacheable. Unallocated space is a space that is not used in a process run, and allocated space is the space used in the process. The cached space is the space that has been loaded into physical memory, and the non-cached space is not yet loaded into physical memory. During process creation, a page table (stored in physical memory) is created for the process and is used for mapping between virtual memory and physical memory, as shown in the following figure.

The page table is cached in physical memory, as can be seen from the diagram above, where the page table should contain at least these parts: a virtual page that points to the assignment, whether the virtual page is buffered, and if it is cached, the mapping between the virtual page and the physical page. Therefore, when the CPU needs to read the data, the following two scenarios occur.

(1) Hit: When the CPU needs to read the data in the VP2, the related hardware queries the page table, and when it finds that the flag bit in the page table that points to VP2 is set to cached (the image above is set to 1), it reads the physical memory directly.

(2) Page fault: When the CPU needs to read the data in the VP3, it will find that the flag bit of the part of the pages table that points to VP3 is not cached, that is, the VP3 is not cached into the physical memory, and a fault occurs. A page fault exception will cause the response in the kernel to be a missing pages handler, which will select one that has been slowed to the physical memory as a sacrifice page, copy it back to disk, and then need the cached virtual page (VP3) cache (copy) to this sacrifice page. Assume that VP4 is set to the Sacrifice page, and the result of the pages processing is shown in the following figure.

When the kernel program loads the contents of the page fault, it re-invokes the instruction that caused the missing pages, which re-sends the virtual address that caused the pages to the address translation hardware, when the VP3 has been loaded into physical memory, and it can be executed correctly. As can be seen from the previous narrative, the page faults will result in the replication between the memory and the disk, if the process in the process of execution, produced a number of pages (called "bumps"), will cause the process of execution speed and low. Although in general, the implementation of the program is local, that is, the program is often on a smaller page to work, so as to ensure that the probability of the bump is lower, but if the program is working in a large page, the program is likely to work in the "bumpy" state, the page is frequently swapped in and out, resulting in serious stuttering.

As can be seen from the above discussion, the operating system provides a separate page table for each process, which is a separate virtual memory space, which greatly protects the process memory from being accessed by other processes because virtual memory is mapped to different areas in physical memory. But sometimes there is a special case where the virtual memory of a different process is mapped to the same piece of physical memory, and we call this physical page "shared Page". In some cases, it is still necessary for the process to share code and data, for example, each process must have the same kernel code, each C program calls the program in the C standard library, and so on. In addition to providing better security, virtual memory provides a simplified memory model. For example, when we request a piece of memory in the program (MALLOC/NEW), we allocate a contiguous block of memory on the virtual memory space. Contiguous blocks of memory can greatly simplify our programming, but in reality the contiguous memory blocks allocated in virtual memory are mapped to physical memory without the need to guarantee their continuity. That is, suppose we request a continuous space of K VP size in virtual memory and map it to K-any PP in physical memory. Due to the way the page table works, there is no need for the operating system to allocate k consecutive pp,pp that can be randomly dispersed over physical memory. At the same time, thanks to the way the page table works, we can even request space that is larger than the physical memory (theoretically, but not infinitely large, that is, larger than physical memory; Under VS2013, the native 8g memory (the remaining 4g is available), the maximum amount of 10G of memory space can be requested).

Second, the translation process of virtual address to physical address

The process of reading data from physical memory as shown above, this is a simplified version, ignoring a lot of details (the book has a more detailed introduction, but also just from the face of simple familiar). Here only write down the process of hit, the process of missing pages more than hit a fault response process. When the CPU needs to read the data, it first generates a virtual address, then looks up the page table based on the virtual address, obtains the actual physical address, and the CPU obtains the actual physical address and reads the data from the physical memory.

In this case, only one page table is drawn, in fact the page table can be multi-level. A two-level page table is shown in the following figure.

For a 32-bit system, using a single-level page table, if each item in the page table can maintain a 4k virtual page table VP, and each item requires 4 bytes, then for 4GB of virtual memory needs 4MB page table in physical memory, if there are many processes, this is almost unacceptable. For a Level Two page table, the first-level page table in the two-page table can maintain a 4M of space for each item, and a second-level page table in a two-page table can maintain 4K of space. If you store only the first-level page table in physical memory, you can greatly reduce the space required for the page table.

The benefits of using the Level two page table are as follows: (1) If a PTE in a page table is empty, then the two-page table does not exist, which represents a significant potential savings because the 4GB virtual address space is largely unassigned for a generic program. (2) Only one level page table in main memory, virtual memory is created when needed, page calls or two-level page table, reduce the pressure of main memory, and can be used to cache the second-level page table in main memory.

However, not more than the number of page table, the more the better, actually with a multi-level page table is not necessarily better than the first page table, or even much slower.

Third, other

Csapp describes the memory allocation and release of the operating system in the following sections of this chapter, which is not covered here. But for a more in-depth discussion of the memory management mechanism, I think it is still to study the kernel of the operating system, and I want to read the book "Garbage Collection Algorithm Handbook: The Art of Automatic memory Management" (Watercress score good, and so after brushing Csapp after reading-_-\\\)

Finally, the first blog reading experience, if the understanding of inappropriate, welcome criticism advice. This part of the best exercise is to write a simple memory pool/memory management, so have time to write a try it.