Linux virtual Memory Related introduction

What is virtual deposit? Why do you need it?





we know that program code and data must reside in memory to be able to run, however, the system memory is very limited, often can not accommodate a full program of all the code and data, not to mention in the multitasking system, you may need to open the subroutine, paint programs, browsers and many other tasks, want to let memory in Leaving all these procedures is obviously unlikely. So the first thing you can think of is to split the program into smaller pieces, just let the current system run all the parts that it needs to stay in memory, and the rest of it will stay on the hard drive. When the system finishes processing the current task fragment, it is then transferred from the external memory to the next task fragment to be run. Indeed, this is how older systems handle big tasks, and the job is done by programmers themselves. But as programming languages become more advanced, programmers are less reliant on system systems, and few programmers are very clear about the system, so letting programmers be responsible for fragmentation and on-demand reduction reduces efficiency, while heavy causes the machine to crash; One reason is that as the program grows richer, The behavior of the program is almost impossible to predict accurately, and it is difficult for programmers to judge which program they need to load next. Therefore, it is difficult to rely on the predictive static allocation of fixed size of memory, and then mechanically rotate the program into memory execution. The system must adopt a new technology that can be allocated on demand without the need for programmer intervention.





virtual memory, known as virtual memory, is relative to logical and physical memory in the system, and logical memory is the memory that is viewed from the process perspective, and is therefore something that programmers care about. While physical memory is the memory that is seen from the processor's perspective, the operating system is responsible for managing it. Virtual memory can be said to be a technical means of mapping to these two different viewing memory. Technology is an on-demand dynamic memory allocation method taken over by the operating system, it allows the program to unknowingly use more than the actual physical space size of the storage space (in fact, the program needs to store the storage space in the form of the spread of storage on the physical memory and disk), so that virtual memory completely freed the programmer, Since then programmers do not care too much about the size and loading of the program, you can freely write programs, cumbersome things to the operating system to do it.





Implement virtual memory


Virtual memory is a combination of the system's hard disk space and the actual memory of the system for use by the process, providing a much larger virtual space than memory. When the program is running, as long as a small part of the virtual address space mapped to memory, the rest are stored on the hard disk (that is, the program virtual space equals the actual physical memory plus part of the hard disk space). When the virtual address being accessed is not in memory, the address is not mapped to memory, but is stored on the hard disk, so the desired virtual storage address is transferred to memory, and when the system is in a tight state, the virtual storage space that is not currently in use can be swapped out to the hard disk to free up physical memory space. The system works so round and round--swapping in, swapping out, and the user is barely able to look at it, thanks to the virtual memory mechanism.





Linux's swap partition is the space that the hard disk specifically reserved for the virtual storage space. Experience should be about twice times the size of memory. If you are interested you can use Swapon-s to view the swap partition size.





great reason is very good to understand, nothing more than the use of memory and hard disk space into virtual memory space. However, the repeated operation of the address map (virtual address mapping to physical address) and virtual address swap is worth careful scrutiny. How does the system actually map the virtual address to the physical address? How can memory constantly swap with the hard drive to swap out the virtual address?




Can the
use the paragraph mechanism to answer the above questions? The logical address changes to a 32-bit address through a segment mechanism, enough to cover 4G of memory space, when the program needs the virtual address is not in memory, only rely on the segment mechanism is difficult to swap the virtual space, because it is not convenient to the entire size of the virtual space in memory and hard drive between the transfer (old system, will be clumsy to swap out the entire memory or even the whole process, think of this will have the consequences of it! ）。 So it is necessary to look for a smaller and more flexible storage representation unit, so that the virtual address can be transferred between the hard disk and memory. This smaller storage management unit is the page (4K size). The mechanism for managing pages to be swapped out is called the page mechanism.





Because the page mechanism is used, the address that is converted through the segment mechanism is only an intermediate address--the linear address, which does not represent the actual physical address, but represents the virtual space address of the entire process. On the basis of the linear address, the page mechanism will then process the linear address mapping: when the desired linear address (virtual space address) is not in memory, the required virtual memory is transferred from disk in page units, and when there is not enough memory, the memory virtual space is swapped out to disk in the page. Visible, the use of pages to manage memory and disk (virtual memory) greatly facilitates the work of memory management. There is no doubt that page mechanism and virtual memory management are simply "perfect".





uses the page mechanism, 4G space is divided into 2 20-time 4K-size pages (the page can also be set to 4M size), therefore, each index entry in the Index Table (page table) that is required to locate the page requires at least 20 bits, but some page properties are often added to the page table entry, so the page table entry is actually 32 bits. 12 bits are used to store information such as "whether a page exists in memory" or "permissions on the page."





before we mentioned the linear address is 32 bits. Its high 20-bit is the index of the page table, and the low 12 bits give the offset in the page. The linear address after the page table to find the page base site and the lower 12-bit offset added to form the final required physical address.





in practice, not all page table entries are stored in a large sheet, because each page table entry occupies 4 bytes and requires 4M of contiguous storage if you want to store 2 of the 20-page table entries in one table. Such a large continuous space is not easy to find, so often will be the page table hierarchical storage, such as two levels, then each page table only need 4k contiguous space.





Level two page table search as you can see novels, find out in which chapter, and then find the section under that chapter. The specific process to look at the following figure:





to sum up, address conversion work requires two technologies, one is the segment mechanism, the other is the page mechanism. The segment mechanism deals with the mapping of the logical address to the linear address, and the page mechanism is responsible for mapping the linear address to the physical address. The two-level mapping completes the arduous task of translating from a logical address seen by a programmer to a physical address seen by the processor.





You can compare these two mechanisms to an address conversion function, where the variable of the mechanism is a logical address, and the function value is a linear address; the variable of the page mechanism is the linear address, and the function value is the physical address. The address translation process is shown below.





Logical Address-(segment function)--> linear address-(page function)--> Physical address.





Although the section mechanism and the page mechanism are involved in mapping, but they are different division of labor, and mutual independence and non-interference, each other do not have to know whether the other side exists.





below we take a look at the Linux example briefly to see how the paragraph page mechanism is used.




segmentation strategy in
Linux

The
segment mechanism is limited in Linux and is not fully utilized. Each task does not arrange separate data segments separately, code snippets, but only minimal use of the segment mechanism to isolate user data and system data--linux only four ranges of segments, kernel data segments, kernel code snippets, user data segments, user code snippets, which cover 0-4g space, The difference is that each section of the attribute is different, the kernel segment privilege level is 0, the user segment privilege level is 3. This section avoids the conversion step of the logical address to the linear address (the logical address is equal to the linear address), but still retains the level of the paragraph of the most basic protection.





Each user process can see the 4G size of the linear space, where 0-3g is User space, user-state processes can be directly accessed from the 3G-4G space for kernel space, storage kernel code and data, only the kernel-state process can directly access, user-state process can not directly access, You can enter kernel space only through system calls and interrupts, and this is the privilege switch that is going to take place.





when it comes to privilege switching, you can't leave the concept of task door, trap door/interrupt door. Trap doors and interrupt doors are channels that enter the kernel space in the event of traps and interruptions. Call Gate is a user space program to access the channel, the task door is more special, it does not contain any address, but services to task switching (but Linux task switching does not really use it, it is too troublesome).





For various gate systems there will be a corresponding gate descriptor, similar to the segment descriptor structure, and the door descriptor is indexed by the corresponding door selector, and eventually a pointer to the offset address within a particular segment is generated. This pointer is about the entrance that will be entered. The purpose of using the door is to ensure that the entrance can be controlled, not into the kernel should not access the location.




paging strategy in
Linux


look at how paging is used in Linux.




Each process in
Linux has its own page tables, which means that the process's mapping functions are different, ensuring that each process virtual address does not map to the same physical address. This is because processes must be independent of each other, and their data must be isolated to prevent information leaks.





need to be aware that the kernel, as a separate part that must be protected, has its own separate page table to map kernel space (not all space, just the size of the physical memory), and the page table (SWAPPER_PG_DIR) is statically allocated to map kernel space only (swapper_pg_ Dir uses only 768 items later--768 page catalogs can map 3G space. This independent page table guarantees that the kernel virtual space is independent of other user program space, that is, other processes are usually not connected to the kernel (when the kernel is compiled, the kernel code is designated to link to more than 3G of space), so the kernel data is naturally protected.





So what do you do when a user process needs to access kernel space?





Linux employs an ingenious approach: the first 768 mapping process space in the User Process page table (<3g, since only the base address is specified in Ldt, only to 0xc0000000), if the process is to access kernel space, such as calling system calls, The table entry after 768 items in the page directory of the process will point to the item after the 768 items in the Swapper_pg_dir, so once the user falls into the kernel, it starts using the kernel's page table Swapper_pg_dir, which means that the kernel space can be accessed.