Thoughts on Linux memory management

From: http://bbs.chinaunix.net/thread-1925077-1-1.html

Classic! Read the low-end physical memory and high 3

 

The virtual memory of the kernel is continuously mapped to the physical memory at the lowest end. This is the beginning of all problems.

Why do we need to continuously map the virtual address space of the kernel to the lowest-end physical memory? This is not a problem at all. This design is done by developers for efficiency or implementation reasons. However, this design raises many confusing problems.

Suppose we use the 32-bit kernel, and the system is equipped with 2 GB physical memory. The "virtual memory of the kernel" mentioned below refers to the first MB virtual memory of the kernel.

"How does the kernel continuously map its virtual memory to the low-end physical memory? Is the low-end physical memory M occupied by the kernel and cannot be used separately? I gave a negative answer in my post discussing memory yesterday. It looks like this is a very old question.

There is often a saying in the book: the kernel can only use the lowest-end 896m physical; Memory larger than 896m is called "high-end memory" and cannot be directly used by the kernel. When a beginner reads this, he always feels that something is wrong, but he does not know why.

Since the kernel has mapped its virtual memory, what is entries in the kernel page table? In the entry, we are currently interested in two items: the ing address and the present flag. The ing address must have been arranged. Is the present flag 1? Consider it as 1. In this way, from the kernel page table, the physical memory of the first MB is indeed occupied by the kernel, as long as the kernel references the corresponding virtual address, the kernel will access the corresponding physical address without any exceptions. However, as mentioned above, the process can also use this memory. If a virtual address of a process is mapped to a segment of the first MB, the kernel and process can access this memory through their respective page tables. Will this cause a conflict?

The data structure of the description page frame is the page structure. Whether a page frame is idle depends on the reference number in the corresponding page structure. So is the 896m physical memory mapped by the kernel idle? In addition to the physical pages occupied by kernel code and static data, all other pages are idle. That is to say, the memory management subsystem can allocate these memories. In yesterday's post, "ing by the kernel does not mean being used by the kernel" is the case. The content in the kernel page table indicates that the first M physical memory is in a ready to use state, but the memory is not used by the kernel, and the process can also be used. In the M memory, the kernel cannot use the memory used by the process, that is, the kernel cannot reference the content in the corresponding page table, that is, the kernel cannot use the corresponding virtual address.

How does the kernel use its own virtual address? How does she know which segment of the virtual address has been used by herself or cannot be used because of a process? That is, how the kernel manages its own virtual address.

First, let's look at how the kernel manages the virtual addresses of processes. When a process requires memory, the kernel first allocates a virtual address (memory region) to the process. Because the usage of the virtual address of the process is recorded in vm_area_struct, all vm_area_struct are connected to a linked list in sequence. Therefore, it is very easy to find a virtual address of a certain size and scan the linked list, when a continuous address greater than or equal to the size of the requested memory is encountered, it is recorded with the new vm_area_struct and inserted to the proper position of the linked list, this virtual address is marked as "used. In fact, this is the first fit in the operating system theory. In the early days of UNIX, physical memory was managed in this way, and the virtual memory of processes was used here. The steps for the kernel to allocate memory to processes are as follows:
1. Find the appropriate virtual address segment;
2. Apply for a physical page frame from {memory management subsystem;
3. Create a ing between the two in the process page table.

However, when the kernel allocates memory for <self>, only step # is available #.

There is no third step because the kernel page table has been mapped at the beginning. What is the first step? Why is the kernel not <find a virtual address segment of the proper size> like a process?

When the kernel itself needs memory, it always applies to the slab layer (?). A new data structure needs to be created. When a buffer is required, slab allocates a continuous memory for it. The kernel does not need to map this memory. As long as this continuous physical memory is within MB, it is in a state that is "ready to be used by the kernel. After being split from slab, it is "officially used by the kernel ". In the end, how does the kernel manage virtual addresses? The kernel does not have "additional management". The specific virtual memory used depends on the physical memory allocated by slab. The physical memory allocated by Slab depends on {memory management subsystem }. {Memory management subsystem }={ buddy system}

Conclusion: the buddy system manages the virtual memory of the kernel. Strictly speaking, the buddy system manages the virtual memory of the kernel at will when managing the first MB of physical memory. For processes, virtual memory allocation and physical memory allocation are separate steps; for the kernel, physical memory allocation equals to virtual memory allocation. The location of the memory allocated by the kernel for the virtual memory depends on the location of the [page frame with configured memory] in the physical memory. The process requires an additional struct to record the usage of the virtual memory. The usage of the kernel virtual memory is the usage of the first MB physical memory. In other words, once a process has applied for a block of the first MB physical memory, it is equivalent to occupying the virtual address space of the kernel, and the virtual address segment corresponding to the kernel has been used. Since all applications for physical memory are answered by the buddy system, it is impossible for the kernel to obtain the [virtual address] corresponding to the [physical memory occupied by the process] When applying for memory for itself. in this way, it is impossible to [reference the corresponding page table], and conflicts are avoided. Of course, if there is a bug in the kernel and the pointer is used carelessly, it is easy to destroy the memory of the process.

In this way, the problem is solved. The first M physical memory is mixed by the kernel and process. But the kernel can only use the M physical memory (because its page table ing is fixed), and the process can use the physical memory anywhere (because its page table can be set at will ). The kernel height is MB for other purposes and can be mapped at will. Memory that cannot be directly used by the kernel is "high-end memory ". When the linear address space of the kernel is larger than the physical memory, "high-end" does not exist. In this case, there are two possibilities: first, the Kernel linear address space is too large, such as using a 64-bit system; second, the physical memory of the system is too small, such as the machine with only MB of memory. Both are results of {Kernel linear address space greater than physical memory.

When managing the virtual memory of a process, you can map the virtual address to any physical address at will using pagination. This is the general usage of paging mechanism ". On the contrary, when the kernel manages its own virtual memory, it uses pagination to fix "one virtual memory to one physical memory", instead, it limits the locations in the physical memory used by the internal nuclear energy, this is contrary to the conventional paging mechanism in our impressions. There is always a kind of inconsistency that cannot be clearly explained for beginners. This is probably the reason.