The caching mechanism of Linux and the method of cleaning Buffer/cache/swap

(1) Caching mechanism

In order to improve the performance of the file system, the kernel uses a portion of physical memory to allocate buffers for caching system operations and data files, and when the kernel receives read-write requests, the kernel first goes to the buffer zone to find out if the requested data is available, directly returns it, and if not, operates the disk directly through the driver.

Caching mechanism Advantages: Reduce the number of system calls, reduce CPU context switching and disk access frequency.

CPU Context Switching: The CPU gives each process a certain amount of service time, and when the time slice is exhausted, the kernel reclaims the processor from the running process, saves the current running state of the process, and then loads the next task, which is called context switching. is essentially a process switch that terminates the running process and the process to be run.

(2) View buffers and memory usage

[Email protected] ~]# free-m

Total used free shared buffers Cached

mem:7866 7725 141 6897

-/+ buffers/cache: 752 7113

Swap: 16382 32 16350

Can see a total memory of 8G, has used 7725M, the remaining 141M, a lot of people are so see, this does not serve as the actual usage. Because of the caching mechanism, how to calculate the specific?
Free memory =free (141) +buffers (+cached) (6897)
Used Memory =total (7866)-free memory
This calculates that the free memory is 7112M, has used memory 754M, this is the real use rate, also can refer to-/+ Buffers/cache This line of information is also the correct memory usage rate.

(3) The visible buffers are divided into buffers and cached, what difference do they have?

The kernel allocates the buffer size to ensure that the system can normally use physical memory and data volume reading and writing. buffers is used to cache metadata and pages, which can be understood as a system cache , for example, vi to open a file. cached is used to cache files, which can be understood as data block caching , for example, DD If=/dev/zero of=/tmp/test count=1 bs=1g Test writes a file, it is cached into the buffer, The next time you execute this test command, the write speed will be significantly faster.

(4) What do you do with swap?

Swap means the swap partition, which is usually what we call virtual memory, which is a partition that is partitioned from the hard disk. When the physical memory is not enough, the kernel releases some long-unused programs in the cache (Buffers/cache) and then temporarily puts them into swap, which means that swap is used if the physical memory and buffer memory are not enough.

Swap cleanup:
Swapoff-a && swapon-a
Note: This cleanup has a precondition that the free memory must be larger than the swap space already in use

(5) How to release buffer memory?
A) direct change of kernel operating parameters
#释放pagecache
Echo 1 >/proc/sys/vm/drop_caches

#释放dentries和inodes
Echo 2 >/proc/sys/vm/drop_caches

#释放pagecache, Dentries and Inodes
Echo 3 >/proc/sys/vm/drop_caches

b) can also use SYSCTL to reset kernel operating parameters

Sysctl-w vm.drop_caches=3

Note: Both of these methods are temporary in effect , the permanent effect needs to be added in the sysctl.conf file , generally written script manual cleanup, it is recommended not to clean.
Modify/etc/sysctl.conf Add the following option and no memory continues to increase
Vm.dirty_ratio = 1
Vm.dirty_background_ratio=1
vm.dirty_writeback_centisecs=2
Vm.dirty_expire_centisecs=3
Vm.drop_caches=3
Vm.swappiness =100
vm.vfs_cache_pressure=163
vm.overcommit_memory=2
VM.LOWMEM_RESERVE_RATIO=32 32 8
Kern.maxvnodes=3

The above settings are rough, so that the role of the cache is basically unable to play. It is necessary to adjust the condition of the machine to find the best compromise.

The caching mechanism of Linux and the method of cleaning Buffer/cache/swap