Storage Management (1)

Storage Management is an important part of the operating system. The Linux operating system uses the request paging virtual storage management method. The system provides 4 GB virtual memory space for each process. The virtual memory of each process is independent of each other. When Linux runs in the X86 architecture, the virtual memory of the process is 4 GB. The division of process virtual memory space is determined by gdt during system initialization. Linux storage management is mainly used to manage the user area of the virtual memory of processes. The virtual memory of a process is divided into code segments, data segments, stacks, environment variables for running processes, and parameter transfer areas. Each process has an mm_struct struct to define its virtual user zone. The first address of the mm_struct struct is in the task struct task_struct member mm. The virtual storage area is a continuous area in the virtual storage space. The information in this area has the same operation and access features. Each virtual area is described using a vm_area_struct struct. The ing of the virtual memory space and the virtual memory (Virtual Memory) created in the virtual storage area allow the computer to manipulate a larger address space, and each process in the system can have its own virtual address space. These virtual address spaces are completely isolated from each other, so the process running an application will not affect other processes. In addition, the hardware virtual memory mechanism allows write protection for the memory area. This prevents code and data from being overwritten by incorrect programs. Memory ing can map the virtual address space of the CPU to the physical memory of the system. The core shared virtual memory mechanism allows the process to have a separate (virtual) address space, but sometimes requires shared memory between processes. For example, the system may have multiple processes running commands to explain the bash program. Although the virtual address space of each process can have a bash backup file, it is better to have only one backup file in the physical memory, all processes running bash share the code. A Dynamic Link Library is another common example of code execution shared by multiple processes. In addition, shared memory is often used for inter-process communication. Two or more processes can exchange information through the shared memory. Linux supports the shared memory IPC Mechanism of System V. Linux virtual memory uses the Dynamic Address ing mode, that is, the correspondence between the address space of the process and the storage space is implemented during the execution of the program. The process uses a virtual address. Therefore, it needs to use MMU to convert the virtual address to the physical address of the memory for access to each address. Dynamic Address ing allows Linux to achieve dynamic migration of processes in the primary storage, dynamic expansion and movement of virtual storage segments, and also provides the basis for the implementation of virtual storage. When running a process image in Linux, you need to call the executable image content. Instead of directly transferring the data to the physical memory, you only need to put the data into the virtual memory area of the process. The memory is actually transferred only when the execution requires the data. The connection between the process image and the virtual process space is called a memory image. When you need to call the process image to the virtual memory space of the process, you need to apply for a suitable virtual memory space. In this case, you need to call the MMAP system to obtain the required memory space. Memory space/address type CPU address space (address space) is an address range that represents the memory location. In X86 architecture, there are three types of address space: physical address space, linear address space, and logical address space (virtual address space ). In the ARM architecture, only the physical address space and logical address space (virtual address space) are available ). A physical address is a real hardware address available in the system. For example, a system has 128 MB of memory, and its valid address range is 0 ~ 0x8000000 (in hexadecimal format) each address corresponds to a group of transistors in the Simms installed and corresponds to a group of specific signals on the processor address bus. The logical address is the total address space that the CPU can process. For a 32-bit CPU, the logical address space is 4 GB. The advantage of using the logical address space is that each user process has its own operating space, instead of having to worry about its actual location in the physical memory. In Linux, the 4 GB address space is shared by the Linux kernel and Linux applications. User space uses 0x00000000 ~ 0xbfffffff contains 3 GB address space, which can be directly accessed by user-mode processes. The kernel space uses 0xc0000000 ~ The remaining 1 GB address space of 0xffffffff stores the code and data accessed by the kernel, and user-mode processes cannot be accessed directly. User processes have the right to access the core State only when they are interrupted or the system calls enter the core State. The conversion between logical and physical addresses is completed by the CPU Memory Management Unit MMU. The Linux kernel is responsible for telling MMU how to map logical pages to physical pages. Generally, the kernel needs to maintain the logical address and physical address table of each process. When switching a process, it updates the MMU table information. MMU automatically performs actual address translation when a process initiates a memory request. In the X86 architecture, linear address ing to physical address is divided into two steps. The linear address provided to the process is divided into three parts: one page Directory Index, one page table index, and one offset. The page Directory is an array of pointers to the page table. The page table is an array of pointers to the page. Therefore, address ing is a process of tracking the pointer chain. A page Directory can determine a page table and then obtain a page. The offset in the page can be used to indicate an address on the page. To provide a more detailed and accurate description, the page Directory items in the given page Directory Index store a page table address stored in the physical memory, the page table items in the given page table index store the base address of the corresponding physical page on the physical memory, and then the offset of the linear address is added to the physical address to form the destination address of the final physical page. The paging mechanism and memory management of MMU Linux adopt paging management. Multi-level page tables are used, and dynamic address translation mechanisms work with primary and secondary storage to implement virtual memory. Each user process has a virtual address space of 4 GB. The process can be dynamically scaled during running, providing users with transparent, flexible, and effective memory usage. This is one of the reasons why a process is allocated a logical address space. Even if each process has the same logical address space, the corresponding physical addresses are also different through the paging mechanism, so they do not physically overlap with each other. From the kernel perspective, both the logical and physical addresses are divided into fixed-size pages. Each valid logic page is located in a physical page to facilitate MMU address conversion. When address translation cannot be completed (for example, the given logical address is invalid or the logical page does not have a corresponding physical page), MMU will interrupt and send a signal to the core. Linux core can handle this page error (page fault) problem. MMU is also responsible for enhancing memory protection. When an application tries to perform write operations on a page marked as read-only in its memory squadron, MMU will also generate an interruption error and notify the kernel. Without MMU, the kernel cannot prevent a process from illegally accessing the memory space of other processes. Each process has its own page Directory and page table. In this process, the base address of the page Directory is the key, and the physical address corresponding to the logic can be found through it. The base address of the page Directory is the private resource of each process. It is stored in the mm_struct Structure Variable mm of the task_struct object of the process. During process switching, the CPU will fill in the base address of the page Directory of the new process with the page Directory register of the CPU for MMU to use. When a new process has an address for access, MMU finds the base address of the page table from the page Directory based on the top 10 digits of the accessed address, then, find the corresponding physical address header from the page table based on the next 10-bit height, and find the real physical address corresponding to the logical address based on the remaining 12-bit offset and the header address. Cache Linux uses many memory management policies related to cache. 1. buffer zone the cache contains data buffers driven by Block devices. These buffer units are generally fixed in size and contain information blocks read or written from Block devices. A block device is a device that can only read or write data in a fixed size block. All hard disks are Block devices. You can use the device identifier and the required block number as an index to quickly find data blocks in the buffer cache. Block devices can only be accessed through the buffer cache. If the data can be found in the buffer cache, it does not need to be read from the physical block device (such as the hard disk), which can accelerate the access speed of the device. The two-page high-speed cache is used to accelerate the access to executable image files and data files on Block devices. It caches the file content of a page each time. The page is read from the block device into the memory and placed into the page cache. 3. The swap cache stores only modified pages in the swap file. If these pages are not modified after they are written to the swap file, the next time they are swapped out of memory, they do not have to be updated or written. These pages can be discarded in systems with frequent exchanges. Switching high-speed cache can reduce unnecessary and time-consuming block device operations. 4 hardware high-speed cache common hardware high-speed cache is the instruction and data cache in the processor. It caches the commands and data recently accessed by the cpu so that the CPU does not need to obtain data from the memory. Cache is a bridge between CPU and memory. Currently, the implementation of cache in common CPUs is classified by read/write mode: 1) This mode separates the cache from the master memory, the CPU sends all data requests from the primary storage to the cache first, and the cache searches for the data by itself. If the request is hit, the request sent from the CPU to the primary storage is cut off and the data is sent. If the request is not hit, the request is sent to the primary storage. The advantage of this method is that it reduces the number of requests sent from the CPU to the primary memory, and the disadvantage is that it delays the access time from the CPU to the primary memory. 2) In this method, when the CPU sends a data request, it does not pass through the cache in a single channel, but sends a request to both the cache and storage. Because the cache speed is faster, if hit, the cache will send the data to the CPU, but it will have time to interrupt the CPU request to the primary storage; if not hit, the cache will not do any action, the CPU directly accesses the memory. It has the advantage that there is no time delay, but the disadvantage is that each time the CPU accesses the primary memory, it occupies part of the bus time. 3) Write mode any write signal sent from the CPU to the cache is also written to the primary storage to ensure that the data in the primary storage can be updated synchronously. It is easy to operate, but because the speed of the primary storage is slow, the system write speed is reduced and the bus time is occupied. 4) Write-back: in order to overcome the disadvantages of reducing the system write speed and occupying the bus time, the primary memory access times should be minimized, there is a write-back statement. Its working principle is: data is generally only written to the cache, so that the data in the cache may be updated and the data in the primary storage will not change (the data is outdated. However, in this case, you can set an ID address and outdated data information in the cache. Only when the data in the cache is changed again, the buyer writes the original updated data to the corresponding unit of the primary storage, then accept the updated data. This ensures that the data in the cache does not conflict with the data in the primary storage.