RHCA RH442 Real-Combat Series (ii) of Linux buffer memory release

1. Description

The kernel of Linux is responsible for the related work of hardware management, resource scheduling, process management, and resource management, in which memory resource management is a very important work for kernel. Kernel when processing a file, such as opening a file, the file metadata information, that is, file name, Inode and other information in the buffer, the subsequent repeated read the same file, then read directly in buffer, such a mechanism to improve speed, in addition, for the contents of the file, Will be recorded in the cache, for buffer and cache, memory will have automatic cleanup mechanism, if buffer and cache has been unable to release, may be caused by: memory leaks, application problems and other reasons.

2. Description of the phenomenon

Production environment, using two large hard disk machine to do glusterfs cluster, for OpenStack Cinder do volume volume role, over time, found that the memory utilization of two machines over 95%, on the machine to view, The utilization of buffer and cache is found to be very high, as follows:

[[email protected] ~]# free -m              total       used        free     shared    buffers      cachedmem:         32057      30861        1195          0           6          29-/+ buffers/cache:          30825        1232            # Buffer has been used close to 30G, leaving only about 1G of space swap:         8191            0       8191 

3. Workaround

1. For buffer and cache too high, use sync to write data back to disk only

[[email protected] ~]# sync && sync && sync[[email  protected] ~]# free -m              total       used       free      shared    buffers     cachedMem:          32057      30854        1203          0           7         29-/+  buffers/cache:          30817        1240             #相比于上次的结果, Freed up 100M of memory, no noticeable effect swap:          8191          0        8191

2. Change the affinity of memory to swap and transfer to swap

[[email protected] ~]# sysctl -a |grep swapvm.swappiness = 60                                                #kernel对于swap的亲和力为60, set to 0, indicates the affinity of swap is adjusted directly using swap space # [[email  Protected] ~]# sysctl -w vm.swappiness=0vm.swappiness = 0[[email protected]  ~]# free -m             total        used       free      shared    buffers     cachedMem:          32057      30857        1200          0           7         29-/+ buffers/cache:       30820       1237                        #依旧没有什么明显的效果Swap:          8191           0       8191# adjusted back [[Email protected] ~]# sysctl  -w vm.swappiness=60vm.swappiness = 60

3. Through the above two methods to try, did not achieve the effect of releasing memory, the above method is more conservative available methods, the following by modifying the kernel parameters of the way to free memory

[[Email protected] ~]# sysctl -a |grep drop_cachesvm.drop_caches = 0                                                #默认为0, which represents the default mechanism # modified to 1, releasing Pagecache, executing several times before execution sync[[email  Protected] ~]# sync && sync && sync && sysctl  -w vm.drop_caches=1vm.drop_caches = 1[[email protected] ~]# free -m              total        used       free     shared     buffers     cachedMem:          32057      30836       1221           0          0          14-/+ buffers/cache:      30821        1236                         #依旧没有明显的内存资源释放Swap:          8191          0        8191# modified to 3, release  pagecache, dentries and inodes[[email  Protected] ~]# sync && sync && sync && sysctl  -w vm.drop_caches=3    [[email protected] ~]# free -m              total       used        free     shared    buffers      cachedMem:         32057        1715      30342           0         12          34-/+ buffers/cache:       1667       30389                        #效果立竿见影!!! , releasing 30G of memory directly swap:         8188           0       8188# change back to original value, remember!! [[Email protected] ~]# sysctl -w vm.drop_caches=0

4. Observe the usage of monitoring memory

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M00/7A/22/wKioL1ai7tPCPg0-AABP2AS_nLA577.png "title=" QQ picture 20160123110725.png "alt=" Wkiol1ai7tpcpg0-aabp2as_nla577.png "/>

4. Parameter description

Drop Caches

Kernels 2.6.16 and newer provide a mechanism to has the kernel drop the page cache and/or Inode and dentry caches on Comm And, which can help free up a lot of memory. Now you can throw away this script that allocated a ton of memory just to get rid of the cache ...

To use /proc/sys/vm/drop_caches, just echo a number to it.

To free Pagecache:

# echo 1 >/proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 >/proc/sys/vm/drop_caches

To free Pagecache, dentries and Inodes:

Echo 3 >/proc/sys/vm/drop_caches

This is a non-destructive operation and would only be free things that is completely unused. Dirty objects would continue to is in the use until written out to disk and is not freeable. If you run the "sync" first to flush them off to disk, these drop operations'll tend to free more memory.

5. Conclusion

About the release of memory, the above through the violent way, directly released the memory of the Inode and Pagecache, about whether to cause data loss, but also in the progress of the observation, if there is a similar situation, it is recommended to restart the process, or to check the application for memory leaks and other issues, as to whether the ability to execute , readers should be cautious and only for reference.

This article is from the "Happy Laboratory" blog, reproduced please contact the author!

RHCA RH442 Real-Combat Series (ii) of Linux buffer memory release