Linux Pagecache and memory consumption

First, Concept introduction

Linux systems typically use the free command to view memory usage, and the free command is primarily read from the/proc/meminfo file and then presented in simple processing, based on the manual of the View Free command to understand the meaning of each field.

650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1032090/201702/ 1032090-20170208005350901-312076513.png "style=" margin:0px;padding:0px;border:0px; "/>

Total: Overall system memory size (min. physical memory mem, swap partition Swap)

Used: Used Memory (Total-free-buffers-cache)

Free: Memory not in use

Shared: Typically the memory used by the TMPFS (memory file system)

Buffers: Memory used by kernel buffers

Cache:page of memory used by the cache and slab

Buff/cache:buffers + Cache

Available: How much memory is available to create a new process without swap. This is different from the number in the free and Buff/cache fields, where the remaining memory is the sum of the values of the page cache, mem slab that can be reclaimed (some slab, cache is in use, not recycled).

The page cache above is a portion of the file that is cached in memory to improve performance when the system reads and writes disk files. This approach improves disk I/O performance, but it also consumes a lot of physical memory, especially when system memory is tight.

Second, the actual use

The Linux system often performs some log writing and backup files, and when these files are large, the corresponding cache consumes a lot of system memory, and these types of caches are not frequently accessed, so the system periodically flush the caches to disk. However, if the system does not flush these caches to disk in a timely manner, it will take up a lot of memory to cause swap, which will affect the overall performance of the system.

/proc is a virtual file system, which can be used as a means of communicating with kernel entities through its read and write operations. This means that you can make adjustments to the current kernel behavior by modifying the files in the/proc. There is a kernel configuration interface ,/proc/sys/vm/drop_caches, which allows the user to manually clean up the cache to free up memory, which has three values: 1, 2, 3. Specifically described below

Writing to this would cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become Free.

-To free Pagecache:

-* echo 1 >/proc/sys/vm/drop_caches

-To free dentries and inodes:

-* echo 2 >/proc/sys/vm/drop_caches

-To free Pagecache, dentries and Inodes:

-* echo 3 >/proc/sys/vm/drop_caches

-as this is a non-destructive operation, and dirty objects be notfreeable, the user should run "sync" first in order to Make sure allcached objects is freed.

-This tunable is added in 2.6.16.

As described above, executing the above command requires the Sync command to flush the page cache to disk and then cache cleanup through the Drop_caches interface. 650) this.width=650; "Src=" http://images2015.cnblogs.com/blog/1032090/201702/ 1032090-20170208005829932-1919569060.png "style=" margin:0px;padding:0px;border:0px; "/>

In addition, when you write code that handles large files, avoid reading the entire file into memory, which can result in a lot of memory, and in extreme cases the normal process is terminated due to an oom of the Linux system. Try to split the file into small files for processing, and close the file as soon as it is finished.

Linux Pagecache and memory consumption