Linux Memory Management Overview (segmented paging mechanism) __linux

"Linux" deep understanding of memory management in Linux

Topics: Segmentation and Paging technology in Linux memory management

Looking back at history, in earlier computers, programs ran directly on physical memory. In other words, it's the physical address that the program accesses during running.

If this system runs only one program, so as long as the memory of this program does not exceed the physical memory of the machine will not be a problem, we do not need to consider the memory management of the trouble, anyway, you have a program, so little memory, eat not enough to eat it is your business.

However, now the system is to support multitasking, multiple processes, so that CPU and other hardware utilization will be higher, this time we need to consider how the system's limited physical memory in a timely and efficient allocation to multiple programs, the thing itself is called memory management.

Give an example of memory allocation management in an early computer system to facilitate understanding.

If we have three programs, program A,B,C, program a running process requires 10M of memory, program B to run the process of need 100M memory, and program C running process needs 20M memory.

If the system needs to run programs A and b at the same time, the early memory management process is probably the same as allocating the first 10M of physical memory to a, and the next 10m-110m is assigned to B.

This method of memory management is relatively straightforward, well, let's say we want program C to run at this time, and assuming that our system has only 128M of memory, it is obvious that the program C is not able to run because of insufficient memory.

You know you can use the technology of virtual memory, memory space can not be used to exchange data to disk space, has reached the purpose of expanding the memory space.

Here's a look at some of the more obvious problems with this memory management approach. process address space cannot be isolated

Because the program accesses the physical memory directly, this time the memory space used by the program is not isolated.

For instance, as mentioned above, the address space of a is 0-10m in this range, but if a section of code in a is manipulating the data in the address space of 10m-128m, then program B and program C are likely to crash (each program can access the entire address space of the system). Such a lot of malicious programs or Trojans can easily break other programs, the security of the system will not be protected, which is also intolerable to users. low efficiency of memory usage

As mentioned above, if we want to run programs A, B, C at the same time, the only way is to use virtual memory technology to write some programs temporarily unused data to disk, and then read the memory from the disk when needed.

Here program C to run, the a switched to disk is obviously not, because the program is the need for continuous address space, program C requires 20M of memory, and a only 10M space, so need to switch program B to disk, and B full 100M, You can see that in order to run program C we need to write 100M of data from memory to disk, and then when program B needs to run and then read from disk to memory, we know that IO operation is more time-consuming, so the process efficiency will be very low. the address of the program running is not sure

Every time a program needs to run, it needs to allocate a large enough free area in memory. The problem is that the idle position is not certain, which will bring some relocation problems, relocation of the problem to determine the program is the reference to the variables and functions of the address, if there is no understanding of child shoes can be looked at the compilation of the principle of information.

Memory management is nothing more than to solve the above three problems, how to make the process of address space isolation, how to improve the use of memory efficiency, how to solve the program when the relocation problem.

Quoting the computer world: "Any problem in a computer system can be solved by introducing a middle tier." ”

The current approach to memory management is to introduce the concept of virtual memory between the program and the physical memory. Virtual memory is located between program and physical memory, the program sees only virtual memory, and no longer has direct access to physical memory. Each program has its own separate process address space, so that process isolation is done. The process address space here refers to the virtual address. As the name suggests, since it is a virtual address, that is, not the real existence of the address space.

Now that we've added a virtual address between the program and the physical address space, we need to figure out how to map to the physical address from the virtual address, because the program is definitely running in physical memory, with two techniques, including segmentation and paging. subparagraph (segmentation)

This method is one of the most popular methods, the basic idea is to map the amount of memory address space required by the program to a physical address space.

　　

Each program has its own virtual independent process address space, and you can see that the virtual address space for both programs A and B starts from 0x00000000. We will map two of the same virtual address space and the actual physical address space one by one mapping, that is, each byte in the virtual address space corresponds to each byte in the actual address space, the mapping process is set by the software mapping mechanism, the actual conversion by the hardware to complete.

The mechanism of this segmentation solves the problem of process address space isolation and program address relocation in the 3 problems that are mentioned in the beginning.

Program A and program B have their own virtual address space, and the virtual address space is mapped to each other not overlapping physical address space, if program a access to the virtual address space is not within the scope of 0x00000000-0x00a00000, then the kernel will reject the request, So it solves the problem of isolating the address space. Our application A only needs to care about its virtual address space 0x00000000-0x00a00000, and it is mapped to which physical address we do not care, so the program always according to this virtual address space to place variables, code, do not need to reposition.

In any case, the segmentation mechanism solves the above two problems and is a big step forward, but there is still nothing to do about memory efficiency problems.

Because this memory-mapping mechanism is still in the program, and when memory is low, the entire program still needs to be switched to disk, so that the efficiency of memory usage is still very low.

So, how to calculate the efficient memory usage. In fact, according to the local operating principle of the program, a program in the process of running, within a certain time period, only a small number of data will be used frequently.

So we need a smaller granularity of memory segmentation and mapping methods, at this point will think of Linux buddy algorithm and slab memory allocation mechanism. Another method of translating virtual addresses into physical addresses has emerged as a paging mechanism. Paging mechanism

The paging mechanism is to divide the memory address space into a number of small fixed-size pages, the size of each page is determined by memory, just as the Ext file system in Linux partitions the disk into several blocks, and this is done to improve memory and disk utilization.

Imagine, if the disk space into n equal parts, each size (a block) is 1M, if I want to store the file on disk is 1K bytes, then the remaining 999 bytes is not wasted. So a finer granularity of disk partitioning is required, we can set the block a little bit, which is of course based on the size of the file stored in a comprehensive consideration, seems a bit off the point, I just want to say, in-memory paging mechanism in the Ext file system with the disk segmentation mechanism is very similar.

The size of the General page in Linux is 4KB, we divide the address space of the process by page, load the commonly used data and code page into memory, and the infrequently used code and data are saved on disk, we still use an example to illustrate the following figure:

　　

You can see that the virtual address space of process 1 and process 2 is mapped to a discontinuous physical address space (this is a significant if one day our continuous physical address space is not enough, but a lot of discontinuous address space, without this technology, our program will not be able to run, Even they share a portion of the physical address space, which is shared memory.

The virtual pages of process 1 VP2 and VP3 are swapped to disk, and the Linux kernel generates a page-fault exception when the program needs two pages, and then the exception manager reads it into memory.

This is the principle of paging mechanism, of course, the implementation of the paging mechanism in Linux is still more complex, through the page global catalog, page Superior directory, page intermediate table of Contents, page tables and other levels of paging mechanism to achieve, but the basic principle of work will not change.

The implementation of the paging mechanism requires hardware implementation, the hardware name MMU (Memory Management unit), which is specifically responsible for translating from virtual addresses to physical addresses, which is to find physical pages from virtual pages.