Improved memory management in Linux2.6 Kernel

Article Title: improved memory management in Linux2.6 kernel. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.
2.6 The Linux kernel uses many technologies to improve the use of a large amount of memory, making Linux more suitable for enterprises than ever before. This article lists some more important changes, including reverse mapping, using larger memory pages, storing page table entries in high-end memory, and more stable memory manager.
　　
With the development and maturity of the Linux kernel, more users expect that Linux can run a very large system to process scientific analysis applications or even massive databases. These enterprise applications usually require a large amount of memory to run properly. 2.4 The Linux kernel has the ability to identify a large amount of memory, but the 2.5 kernel has undergone many changes, enabling it to process more memory in a more effective way.
　　
　　 Reverse ing
In the Linux memory manager, the page tables keep track of the physical pages of memory used by processes. They map virtual pages to physical pages. Some of these pages may not be used for a long time and should be exchanged. However, before they can be swapped out, you must find every process mapped to that page so that the page table entries on the corresponding pages of those processes can be updated. In the Linux 2.4 kernel, this is a daunting task, because to determine whether a page is mapped by a process, you must traverse the page tables of each process. As the number of processes running in the system increases, the workload for switching these pages increases.
　　
Reverse ing, or RMAP, is implemented in the 2.5 kernel to solve the problem. Reverse ing provides a mechanism to discover which processes are using a given memory physical page. Instead of traversing the page tables of each process, the memory manager now creates a linked list for each physical page, including the page table entries pointing to each process mapped to the current page (page-table entries, PTE) pointer. This linked list is called the PTE chain. The PTE chain greatly increases the speed of finding processes mapped to a page, as shown in 1.
　　 　
　　 Figure 1. Reverse ing in 2.6
Of course, nothing is free: it is also costly to use reverse ing to improve performance. The most important and obvious cost of reverse ing is that it brings some memory overhead. We have to use some memory to keep track of all those reverse mappings. Each entry of the PTE chain uses four bytes to store the pointer to the page table entry, and the other four bytes to store the pointer to the next entry of the chain. These memories must use low-end memory, which is not enough on 32-bit hardware. Sometimes this can be optimized to use only one entry instead of a linked list. This method is called the direct p page method (page-direct approach ). If there is only one ing to this page, you can use a pointer called "direct" to replace the linked list. This optimization can be performed only when a page is mapped by a unique process. If the page is mapped by another process later, it will have to use the PTE chain again. A tag setting is used to tell the memory manager when such optimization is effective for a given page.
　　
Reverse ING also brings some other complexity. When a page is mapped by a process, a reverse ing must be established for all those pages. Similarly, when a process releases a page ing, the corresponding ing must also be deleted. This is especially common when you exit. All these operations must be performed when locked. For applications that execute a lot of derivation and exit, this may be a waste and increase a lot of overhead.
　　
Despite some trade-offs, it can be proved that reverse ing is a valuable modification to the Linux memory manager. In this way, the serious bottleneck of finding a process mapped to a page is minimized to only one simple operation. When a large application requests a large amount of memory from the kernel and multiple processes share the memory, reverse ing helps the system continue to run and expand effectively. More improvements to reverse ing are currently under study and may be available in future Linux kernel versions.
　　
　　 Large Memory Page
Typically, the Memory Page processed by the memory manager on the x86 system is 4 KB. The actual page size is related to the architecture. For most purposes, it is most effective for the memory manager to manage memory with pages of this size. However, some applications require a very large amount of memory. Large databases are a common example. Since each page needs to be mapped by each process, you must create a page table entry to map the virtual address to the physical address. If a process uses 4 kb pages to map 1 GB of memory, 262,144 page table entries are used to keep track of those pages. If each page table entry consumes 8 bytes, the overhead of each ing of 1 GB memory is 2 MB. This is already a considerable overhead. However, if multiple processes share the memory, the problem will become more serious. In this case, each process mapped to the same 1 GB memory will pay 2 MB for the page table entries. If there are enough processes, the overhead waste may exceed the memory used by application requests.
　　
One way to solve this problem is to use a larger page. Most new processors support at least one small and one large memory page size. On x86, the size of a large memory page is 4 MB, or 2 MB on the system opened by the physical address extension (PAE. Assuming that the previous page size is 4 MB in large memory pages, the same 1 GB memory can be mapped with only 256 page table entries, without 262,144. In this way, the overhead is changed from 2 MB to 2,048 bytes.
　　
The use of large memory pages can also improve performance by reducing the number of translation index buffer (TLB) failures. TLB is a kind of high-speed cache for page tables, so that the pages listed in the table can be converted from virtual addresses to physical addresses more quickly. Large Memory pages can use fewer actual pages to provide more memory, which is equivalent to a smaller page size. The larger memory pages are used, the larger the memory, the more it can be referenced by TLB.
　　
　　 Store page table entries in high-end memory
On 32-bit machines, page tables can only be stored in low-end memory. The low-end memory is limited to only the first 896 MB of physical memory, while meeting most of the remaining kernel requirements. When applications use a large number of processes and map a large amount of memory, the low-end memory may soon be insufficient.
　　
At present, there is a configuration option in the 2.6 kernel called Highmem PTE, so that page table entries can be stored in high-end memory, release more low-end memory areas for other kernel data structures that must be placed here. At the cost, the process of using these page table entries will be a little slower. However, for systems where a large number of processes are running, storing page tables in high-end memory can squeeze out more memory in the low-end memory area.
　　
　　 Figure 2. Memory Area
　　 Stability
Better stability is another important improvement of the 2.6 Memory Manager. When the 2.4 kernel was released, users almost immediately began to encounter memory management-related stability issues. From the impact of memory management on the entire system, stability is crucial. Most of the problems have been solved, but the solution must fundamentally overturn the original memory manager and rewrite a simple multi-manager to replace it. This leaves a lot of space for Linux publishers to improve their Linux memory manager for specific releases. On the other hand, however, the improvements are that the memory management components in 2.4 are very different because they use different release versions. To avoid this, memory management becomes the most detailed part of kernel development in 2.6. From very low-end desktop systems to large, enterprise-level, multi-processor systems, new memory management code has been tested and optimized on them.
　　
　　 Conclusion
The improvements to memory management in Linux 2.6 kernel are far more than those mentioned in this article. Many changes are subtle but important. These changes, together with the memory manager in the 2.6 kernel, are designed to achieve higher performance, efficiency, and stability. There are some changes, such as Highmem PTE and large memory pages, to reduce memory management overhead. Other changes, such as reverse ing, improve the performance in some key fields. These special examples are chosen because they illustrate how the Linux 2.6 kernel is adjusted and enhanced to better handle enterprise-level hardware and applications.