Introduction to Linux memory management mechanism

In Linux often found that the free memory is very small, it seems that all the memory is occupied by the system, the surface of the memory is not enough to use, it is not. This is an excellent feature of Linux memory management, which differs from the memory management of Windows.

The main feature of the

is that regardless of the size of physical memory, Linux is fully utilized to read some program-called hard disk data into memory, using the high-speed features of memory read and write to improve the data access performance of the Linux system. Instead, Windows allocates memory for the application only when it needs memory, and does not take full advantage of the large capacity of the memory space. In other words, the pre-allocated memory mechanism of Linux itself allows the physical memory on the host to be fully used. With each additional physical memory, Linux makes the most of it, leveraging the benefits of hardware investment, and Windows only makes it a device, even if it increases by 8GB or larger. This feature of the

Linux can improve data access performance by leveraging free physical memory to carve out some space as cache and buffers.

1, what is the cache? The

Page cache (cache) is a primary disk cache implemented by the Linux kernel. It is primarily used to reduce I/O operations on disks. Specifically, by caching the data in the disk into physical memory, access to the disk becomes access to physical memory. The value of the

disk cache is two: first, accessing the disk is much slower than accessing the memory, so accessing the data from memory is faster than accessing it from disk. Second, once the data is accessed, it is likely to be accessed again in the short term. The

Page cache is made up of physical pages in memory, and each page in the cache corresponds to multiple blocks on the disk. Whenever the kernel starts to perform a page I/O operation (typically a disk operation on a block of page size in a normal file), it first checks whether the required data is in the cache, and if so, the kernel uses the data in the cache directly, thereby avoiding access to the disk.

For example, when you open a source program file with a text editor, the file's data is transferred into memory. As you edit the file, more and more data is transferred into the memory page. Finally, when you compile it, the kernel can directly use pages in the page cache without having to read the file back from disk. Because users tend to read or manipulate the same file over and over again, the page cache can reduce the number of disk operations.

2, how does the cache update?

The write operation is actually delayed due to caching of the page cache. When the data in the page cache is newer than the data stored in the background, the data is called dirty data. Dirty pages that accumulate in memory must eventually be written back to disk.

Dirty pages are written back to disk when the following two scenarios occur:

When free memory falls below a specific threshold, the kernel must write the dirty page back to disk to free up memory.
When a dirty page resides in memory for more than a certain threshold, the kernel must write the dirty page that is timed out back to the disk to ensure that the dirty page does not reside in memory indefinitely.

In the 2.6 kernel, a group of kernel threads-pdflush the background write-back routines to perform both tasks uniformly.

First, the Pdflush thread flushes the dirty page back to disk when the idle memory in the system falls below a specific threshold. The background writeback routine is designed to free up dirty pages to regain memory when available physical memory is too low. Specific memory thresholds can be set through Dirty_background_ratio sysctl system invocation.

When the free memory ratio threshold: Dirty_background_ratio is low, the kernel calls the function Wakeup_bdflush () to wake up a pdflush thread, and then pdflush the thread further calls the function background_writeout () to start writing dirty pages back to disk. The function Background_ writeout () requires a long integer parameter that specifies the number of pages to be written back. The function background_writeout () writes the data continuously until the following two conditions are met:

The minimum number of pages that have been specified has been written out to disk.
The number of free memory has been picked up, exceeding the threshold value of dirty_background_ratio.

The above conditions ensure that the Pdflush operation can alleviate the low memory pressure in the system. The writeback operation does not stop until both conditions are reached, unless Pdflush writes back all the dirty pages, and no remaining dirty pages can be written back.

To meet the second goal, the Pdflush background routine is periodically woken up (regardless of whether the free memory is too low), writes out dirty pages that have been in memory for too long, ensuring that there are no persistent dirty pages in memory. If the system crashes and the memory is in disarray, the dirty pages that have not yet been written back to the disk in memory are lost, so it is important to periodically synchronize the page cache and disk. at system startup, the kernel Initializes a timer that periodically wakes up the pdflush thread and then makes it run function wb_kupdate ().