Author: fenng | reprinted. The original source and author information and copyright statement must be indicated in hyperlink form during reprinting.
Web: http://www.dbanotes.net/database/linux_outofmemory_oom_killer.html
My colleagues encountered a small problem on the Linux server. I also went up for half a day. This is the first time that we have noticed the features of Linux over the years: OOM killer. To put it bluntly, oom killer is a layer of protection mechanism, which is used to prevent Linux from getting too serious problems when the memory is insufficient, and kill irrelevant processes.
First, you need to learn some old knowledge. Addressing is limited in a 32-bit CPU architecture. The Linux kernel defines three regions:
# DMA: 0x00000000-0x00999999 (0-16 MB))
# Lowmem: 0x01000000-0x037999999 (16-896 MB))-Size: 880 MB
# Highmem: 0x038000000-
Lowmem (also called normal zone) has a total of 880 MB and cannot be changed (unless the hugemem kernel is used ). For high-load systems, oom killer may be caused by poor lowmem utilization. One possible reason is that lowfree is too small, and the other reason is that the lowmem is full of fragments and the request does not reach the continuous memory area. [according to one case I encountered, one Conjecture is that some applications have a relatively large memory size for one-time requests, but within 880 m, idle (lowfree) is not large enough, and OOM killer will be triggered to work ]. Check the current lowfree value:
# cat /proc/meminfo |grep LowFree
Check lowmem memory fragments:
# cat /proc/buddyinfo
The preceding command must be valid in the 2.6 kernel environment. It is said that the sysrq method is better, but it should be used in Hang. See Metalink Note: 228203.1.
According to some documents, oom killer performs differently in 2.4 and 2.6. In version 2.4, the new process (new memory application) is killed. On the other hand, the process that occupies the most memory is killed (this is very dangerous and can easily lead to system application paralysis ).
For "> RHEL4. added the VM. lower_zone_protection parameter. The default unit of this parameter is mb. The default value is 0, and the lowmem value is 16 Mb. We recommend that you set VM. lower_zone_protection = 200 or greater to avoid fragments in the lowmem region. This parameter can solve this problem ).
For "> RHEL3 (kernel 2.4) does not seem to be a good solution. One is to use the hugemem kernel (heaven knows if it will introduce new issues), the other is to upgrade to 2.4.21-47 and use the new core parameter VM. VM-defragment controls the number of fragments. Then use "> RHEL4 (kernel 2.6. To put it bluntly, if OOM killer is encountered, it is basically a defect in the design of the earlier version of kernel.
Others, If you query the RedHat bug library, you will find that many kernel versions also have bugs. Especially when "> NFS.
Tip: Disable and activate OOM killer:
# echo "0" > /proc/sys/vm/oom-kill
# echo "1" > /proc/sys/vm/oom-kill
For more information, see:
- 1) OOM killer "out of memory: killed process" Solutions/Summary]
- 2) Metalink notes: Linux kernel lowmem pressure issues and kernel Structures
- 3) respite from the OOM killer