"Deep understanding of computer system 2nd" Part of Reading notes---9th virtual memory (unfinished)

Process--excerpt from "Deep Understanding of computer system 2nd," 8th chapter of abnormal control flow

The key abstraction that a process provides to an application:

a separate logical control flow that provides an illusion as if our program uses the processor exclusively . a private address space that provides an illusion as if our program exclusively uses the memory system .
These two abstractions are important and helpful in understanding some of the mechanisms described in "Windows core programming 5th" and "the UNIX Environment Advanced Programming 2nd".

Virtual Storage

Virtual memory is the perfect interaction between hardware exceptions, hardware address translation, main memory, disk files, and kernel software, which provides a large, consistent, and private address space for each process. With a clear mechanism, virtual storage provides three important capabilities:

(1) it takes main memory as a cache of address space stored on disk, saves only the active region in main memory, and transmits data back and forth between disk and main memory as needed , and in this way it uses main memory efficiently.

(2) it provides a consistent address space for each process , simplifying storage management.

(3) it protects the address space of each process from being corrupted by other processes .

physical and virtual addressing

Physical Addressing

The main memory of a computer system is organized into an array of m contiguous byte size units. Each byte has a unique physical address (physical ADDRESS,PA). The first byte has an address of 0, the next byte is 1, the next one is 2, and so on. Given this simple structure, the most natural way for a CPU to access memory is to use physical addresses, which we call physical addressing.

Virtual Addressing

When using virtual addressing, the CPU accesses main memory by generating a virtual address (virtual Address,va), which is converted to the appropriate physical address before being sent to memory. The task of translating a virtual address into a physical address is called address translation (translation). Like exception handling, address translation requires close collaboration between CPU hardware and the operating system. The CPU chip is called the Memory Management Unit (Memory Management Unit,mmu) The specialized hardware, uses in the main memory the query table to dynamically translate the virtual address, the table content is manages by the operating system.

address Space

The address space (adress spaces) is an ordered set of non-integer addresses: {0,1,2,...}

If the integer in the address space is contiguous, then we say it is a linear address space (linear). In a system with virtual memory, the CPU generates a virtual address from an n = 2 ^ n address space, which is called the virtual address space: {0,1,2,3,..., N-1}

The size of an address space is described by the multiples required to represent the maximum address. For example, a virtual address space containing a n=2^n address is called an n-bit address space. Now the system typically supports 32-bit or 64-bit virtual address space.

A system also has a physical address space (physical addresss spaces) that corresponds to the M byte of the physical memory in the system: {0,1,2,... M-1}

M does not require a power of 2, but in order to simplify the discussion, we assume M = 2 ^ M.

The concept of an address space is important because it clearly distinguishes between data Objects (bytes) and their properties (addresses). Once this distinction is realized, we can generalize it, allowing each data object to have multiple separate addresses, each of which is chosen from a different address space (discontinuous meaning). )。 This is the basic idea of virtual memory . Each byte in main memory has a virtual address selected from the virtual address space and a physical address selected from the Physical address space. (This paragraph does not know how to understand ~ ~)

Virtual Storage as a caching tool

Conceptually, virtual memory (VM) is organized into an array of n contiguous byte sizes that are stored on disk. Each byte has a unique virtual address, and this unique virtual address is indexed to the array. The contents of the array on the disk are cached in main memory. As with other caches in the memory hierarchy, the data on the disk (lower layer) is partitioned into blocks that act as a transport unit between the disk and main memory (the higher layer). The VM system handles this problem by dividing virtual memory into blocks of fixed sizes called virtual pages (vitual PAGE,VP). The size of each virtual page is P = 2 ^ n bytes. Similarly, physical memory is split into a physical page (physical page,pp), and the size is also a P-byte (the physical page is also called a page frame).

At any one time, the collection of virtual pages is divided into three disjoint subsets:

Unassigned : A page that the VM system has not yet allocated (or created). Unallocated blocks do not have any data associated with them and therefore do not occupy any disk space. (No call to malloc or mmap) Cached : The allocated page in the physical store is currently slow to exist. (already called malloc and Mmap, which is being referenced in the program) Not cached : the allocated pages in physical storage are not slow to exist. (already called malloc and mmap, not yet referenced in the program)

Page Table

Like any cache, a virtual memory system must have some way of determining whether a virtual page is stored somewhere in the DRAM. If so, the system must also determine which physical page The virtual page is stored on. If not hit, the system must determine where the virtual page resides on the disk, select a sacrificial page in physical storage, and copy the virtual page from disk into DRAM to replace the sacrificial page.

These features are jointly provided by many hardware and software, including the operating system software, the address translation hardware in the MMU (memory snap-in), and a data structure stored in physical memory called a page Table (page table), which maps virtual pages to physical pages. A page table is an array of page table entries (ENTRY,PTE).

Linux virtual memory system

Linux maintains a separate virtual address space for each process .

Kernel virtual storage contains code and data structures in the kernel. Some areas of kernel virtual storage are mapped to physical pages shared by all processes. For example, each process shares the kernel's code and global data Structures .

1, linux virtual memory Zone (also concept of area under Windows)

Linux organizes virtual memory into a collection of areas (also called segments). A region (area) is a contiguous slice (chunk) of an already existing (allocated) virtual memory, which is associated in some way. For example, code snippets, data segments, heaps, shared library segments, and different areas of the user stack. each existing virtual page is saved in a zone, and a virtual page that is not part of a zone does not exist and cannot be referenced by the process. The concept of a zone is important because it allows gaps in the virtual address space. The kernel does not have to record virtual pages that do not exist, and such pages do not occupy memory. Any additional resources for the disk or the kernel itself .

The kernel maintains a separate task structure for each process in the system (task_struct in the source code). The elements in the task structure contain or point to all the information that the kernel needs to run the process (for example, PID, a pointer to the user stack, the name of an executable target file, and a program counter).

An entry in task_struct points to Mm_struct, which describes the current state in the virtual storage. PGD points to the base address of the first-level page table (page global catalog), while mmap points to a list of vm_area_struct (regional structures) where each vm_area_structs describes a region (area) of the current virtual address space. When the kernel runs this process, it stores the PGD in the CR3 control register.

A specific area structure contains the following fields:

Vm_start: Point to the beginning of the area. Vm_end: Point at the end of the area. Vm_prot: Describes the read and Write permission permissions for all pages contained within this area. Vm_flags: Describes whether the pages in this area are shared with other processes, or whether the process is private (and some other information is described). Vm_next: Point to the next area structure in the list.


Memory mappings (there are similar mechanisms under Windows, called Memory mappings) Linux (as well as some other forms of Unix) by associating a virtual memory area with an object on a disk (object) to initialize the contents of the virtual memory area, a process called a memory map (memory mapping)。 A virtual memory region can be mapped to one of two types of objects: (1)Common Files on Unix files: An area can be mapped to a contiguous portion of a regular disk file, such as an executable target file. The file Area (section) is divided into page size slices, each one containing the initialization content of a virtual page. Because of the page height on demand, these virtual pages do not actually have physical storage until the CPU first refers to the page (that is, it launches a virtual address and falls within the scope of this page in the address space). If the area file area is large, populate the remainder of the area with zero.
(2)Anonymous Files: An area can also be mapped to an anonymous file, which is created by the kernel and contains binary zeros. The first time a CPU references a virtual page in such an area, the kernel finds a suitable sacrificial page in the physical memory, and if the page is modified, the page is swapped out, with a binary zero covering the sacrifice page and updating the page table to mark the page as residing in memory. Note that there is no actual data transfer between the disk and the storage. For this reason, pages that are mapped to an anonymous file are sometimes called a page that requests binary zeros (Demand-zero page). In either case, once a virtual page is initialized, it is exchanged between a specialized swap file (swap files) maintained by the kernel. Swap files are also called Swap spaces (swap space) or swap areas (swap area). A very important point to be aware of,at any given time, the swap space limits the total number of virtual pages that the currently running process can allocate。
look at shared objects againAn object can be mapped to a region of a virtual store, either as a shared object or as a private object. If a process maps a shared object to a region of its virtual address space, then any writes of the process to the region are visible to other processes that also map the shared object to their virtual memory. Also, this change is reflected in the original object on the disk. (One way of IPC) on the other hand, changes to a region mapped to a private object are not visible to other processes, and any writes that the process makes to the zone are not reflected in the objects on the disk. A virtual storage area mapped to a shared object is called a shared area. Similarly, there are private areas.

The key point of a shared object is that even if the object is mapped to more than one shared area, physical storage needs only one copy of the shared object.

a Shared object (note that the physical page is not necessarily contiguous.) )

private objects are mapped to virtual memory using a clever technique called a write-time copy (Copy-on-write). For each process that maps private objects, page table entries for the corresponding private area are marked as read-only, and the zone structure is marked as a private write-time copy.

and look at the fork function .

When the fork function is called by the current process, the kernel creates a variety of data structures for the new process and assigns it a unique PID. In order to create virtual storage for this new process, it creates the mm_struct of the current process, the region structure, and the original copy of the page table. It reads each page in two processes as a token, and marks each of the two processes as a private write-time copy.

When fork is returned in a new process, the new process now has the same virtual memory as the virtual memory that was present when the fork was invoked. When either of these two processes writes later, the write-time copy mechanism creates a new page, thus maintaining the abstract concept of the private address space for each process.

and look at the EXECVE function .

Suppose a program running in the current process performs the following call:

Execve ("A.out", null,null);

The EXECVE function loads and runs the program contained in the executable target file a.out in the current process, effectively replacing the current program with the A.out program. The following steps are required to load and run A.out:

deletes a user area that already exists . Deletes the existing zone structure in the user portion of the current process virtual address. Map Private areas . Creates a new regional structure for the text, data, BSS, and stack areas of the new program. All of these new areas are private and write-time copies. The text and data regions are mapped to text and data areas in the A.out file. The BSS area is a request for binary zero, mapped to an anonymous file, and its size is contained in the a.out. Stacks and heap areas are also requests for binary zeros. Map a shared area . If a.out programs are linked to shared objects (or targets), such as standard C library libc.so, these objects are dynamically linked to the program and then mapped to a shared area in the user's virtual address space. Set up program counters (PCS). The last thing Execve do is set the program counter in the current process context to point to the entry point of the text area. The next time this process is scheduled, it will begin execution from this entry point. Linux will switch to code and data pages as needed.


user-level memory mapping using the MMAP function

#include <unistd.h>
#include <sys/mman.h>


void *mmap (void *start,size_t length,int Prot,int flags , int fd,off_t offset);
				Return: If successful, it is a pointer to the mapped area, or map_failed (-1) If the error occurs

The MMAP function requires that the kernel create a new virtual memory area, preferably from an area starting with address start, and map a contiguous slice (chunk) of the object specified by the file descriptor FD to the new region. The contiguous object slice size is length byte, starting at the offset byte from the beginning of the file. The start address is merely a hint, usually defined as null.

Munmap function deletes the area of virtual storage:
#include <unistd.h>
#include <sys/mman.h>


int munmap (void *start,size_ t length);
	Return: If the success is 0, if the error is-1

Dynamic Memory allocation
-----------------------is adjourned