Wowo-Understanding of page recycling

Basic concepts of page recycling

Author: Linuxer Released: 2017-8-25 19:01 Category: Memory management

This paper mainly introduces some basic concepts in the page reclaim mechanism. This document can also be viewed as reading a reading note in the first section of chapter 17th of Ulk. Although Ulk has read many times, but every time still feel a harvest. The Linux kernel continues to evolve, but the basic concept of page recycling is constant, so ulk is still worth the careful taste of the kernel enthusiast.

One, what is page frame reclaiming.

In the memory use of user processes, the Linux kernel does not have strict restrictions, in fact, the idea is: when the system load is small, memory is located in various caches, in order to improve performance. When the system is heavily loaded (with a very large number of processes), the memory in the cache is retracted and then used for the creation and mapping of the process address space. Under the guidance of this way of thinking, at first, the kernel was profligate, and had been allocated page frame from the free list of the partner system to the user process or various kernel caches, but the system memory was limited, when the partner system's idle memory dropped to a certain level, The system recycles the page frame to the partner system from the user process or the kernel cache in order to meet the new memory allocation requirements, which is the page frame reclaiming. Word, page frame reclaiming is to ensure that the system is idle in the presence of a specified water level.

Second, when to start Page frame reclaiming.

Can you wait until the free memory is exhausted before you start page frame reclaiming? No, because memory is also consumed in the process of page frame reclaiming (for example, in the process of paging, we may swap the page frame's data to disk, so we need to allocate the buffer head data structure to complete the IO operation), Therefore, we must maintain a certain level of water in the partner system, so that the page frame reclaiming mechanism works properly, so that more memory can be recycled, the system will run, or the system would be crash out.

Third, which scene can trigger page frame reclaiming.

It is the most intuitive idea to trigger page recycling when allocating memory fails, but the kernel scenario is not as simple as allocating memory in an interrupt context, when we cannot trigger page frame reclaiming, because we cannot block the current process at this time. In addition, some memory allocation scenarios occur after holding locks (for exclusive access to certain resources), and we cannot activate IO operations for memory reclamation.

Therefore, summed up, the kernel in memory recycling idea is: the system in allocating memory when the check, if necessary to wake up the KSWAPD, so that the number of free memory system to maintain a level, so as not to push themselves into the desperate. If there is no way, it does go into a desperate situation (allocating memory failure), then directly trigger page recycling. The specific scenarios include:

1, synchronous page reclaim, that is, when encountering the allocation of memory failure, a statement, and then directly call the page frame reclaiming for recycling. For example: When assigning page buffer (alloc_page_buffers), if the allocation is unsuccessful, call free_more_memory for memory reclamation. Or, when calling __alloc_pages, if the allocation is unsuccessful, call try_to_free_pages for memory reclamation directly. Of course, in fact, Free_more_memory is also called try_to_free_pages to achieve page recycling.

2, Suspend to disk (hibernation) scene. When the system hibernate requires a lot of memory, it calls Shrink_all_memory to reclaim the specified number of page frames.

3, KSWAPD scene. KSWAPD is a kernel thread dedicated to page recycling.

4, slab memory allocator will be timed queue work to system_wq up, which will periodically call Cache_reap to reclaim the free memory on slab.

Iv. the policy of page recycling.

First we need to classify page frame into 4 main categories:

1, there is no way to recover the page frame. Including the free page (already on the list, there is no need to bother page frame reclaim mechanism), keep the page (set pg_reserved, such as kernel body segment, data segment, etc.), the kernel dynamically assigned page frame, The page frame on the kernel stack of the user process, the temporary locked page frame (that is, the pg_locked flag is set, for example, when disk IO is in progress), the mlocked page frame (VMA with vm_locked identification)

2. A page frame (swappable) that can be swapped to disk. Anonymous mapping page for user space (user process heap and page frame on stack), Tmpfs page frame.

3. Page frame (syncable) that can be synchronized to the disk. The file mapped page of the user space, and in page cache, page frame (whose contents correspond to a disk file), block device's buffered cache, disk cache page FRAME (for example, Inode cache)

4. Page frame that can be released directly. A variety of memory caches (such as the slab memory allocator) that are not used in the page frame, unused dentry cache.

The second and third classes above are somewhat similar, and the page frame has a fallback file or disk, but we can distinguish that. Swappable page frame, where the final location of the data is memory, the backup file or disk is just to extend the capacity of the memory. Syncable page frame, where the final location of the data is the disk, the memory is simply to speed up. Therefore, when you recycle swappable pages, you need to save the page frame's data to a backup disk or file. And when recycling syncable page, to see whether page frame is dirty, if dirty, you need disk operation, otherwise you can directly recycle.

It is easy to delimit the candidate page frames that are collecting pages (that is, the page frame of the 2, 3, 4 types above), but how do you consider the order in which the pages are recycled? The basic rules for Linux kernel settings are as follows:

1, try not to modify page table. For example, when we recycle a variety of unused kernel caches, we simply recycle them without having to modify the page table entries. Page recycling of user-space processes often involves modifying the corresponding PTEs entry to an invalid state.

2. Unless you call Mlock to lock the page, all user space corresponding page frame should be recyclable.

3. If a page frame is shared by multiple processes, then we need to clear all PTEs entry before we can reclaim the page.

4. Do not recycle page frames that have recently been used (accessed), or prioritize to reclaim page frames that have not been accessed recently.

5. Try to reclaim the page frame that does not require disk IO operations.

Original articles, forwarded please indicate the source. Snail Nest technology, www.wowotech.net.