ORA-600 [KGHLKREM1] problem with Drop_cache when using Linux large page memory configuration
From:
ORA-600 [KGHLKREM1] on Linux Using Parameter drop_cache on hugepages Configuration (document ID 1070812.1)
Suitable for:
Oracle database-enterprise edition-version 10.2.0.1 and later
Generic Linux
Checked for relevance on 29-jul-2013***
Symptoms:
You are running an Oracle database, single-instance db, or RAC. You use the SGA based on large page memory
You encounter the following problem:
Ora-00600:internal error code, arguments: [KGHLKREM1], [0x06bc00020]with stack trace similar to:kghnerror kghadd_reserve D_ext kghgex--or Alsoora-07445:exception encountered:core dump[kglhdal () +1105][SIGSEGV] [Address not mapped to object] [0x000000008] [] []ora-07445:exception encountered:core dump [Kghfnd () +2328] [Sigsegv][address not mapped to object] [0x FFFFFFFFFFFFFFF0] [] []
The SGA heap dump is shown for the violation address (0x6bc00020 in the example above), cleared 0 (zeroed out)
asm1_lmd0_8600.trc~~~~~~~~~~~~~~~~~~*** 2010-02-08 15:57:38.274***** Internal heap ERROR KGHLKREM1 addr=0x6c400020 ds= 0x60000058 ********** Dump of memory around addr 0x6c400020:06c3ff020 00000000 00000000 00000000 00000000 [..... ...] .] Repeat 511 Times
Change:
1.
In your system, you are running Vm.drop_caches=1 (or 3), Drop_cache is set to a value other than 0, or you are performing:
Echo 3 >/proc/sys/vm/drop_caches
The following explanations of drop_caches are not translated, as follows:
/proc/sys/vm/drop_caches (since Linux 2.6.16) Writing to this file causes the kernel to drop clean caches, dentries and Ino Des from memory, causing this memory to become free. To free pagecache:* echo 1 >/proc/sys/vm/drop_cachesto free dentries and inodes:* echo 2 >/proc/sys/vm/drop_caches To free Pagecache, dentries and inodes:* echo 3 >/proc/sys/vm/drop_cachesas The is a non-destructive operation, and D Irty objects is not freeable, the user should run "sync" first on order to make sure all cached objects is freed.
2.
Large pages of memory already using Linux.
Reason:
This is a problem with the Linux kernel
Memory corruption occurs when using the Linux kernel drop_cache parameter and large page memory.
Per internal Bug 9461825, executing vm.drop_caches corrupts Oracle Database SGA hugepages.
This is the fixed in Linux Kernel version 2.6.18-194.0.0.0.4.el5
Solution:
1. As a workaround when hugepages is set avoid any vm.drop_cache settings.-OR-2. Upgrade to Linux Kernel version 2.6.18-194.0.0.0.4.el5
Reference:
Bug:9358381-asm INSTANCE is crashing as ora-600[kghlkrem1] when hugepages
"Translated from MoS article" ORA-600 [KGHLKREM1] issues caused when using Drop_cache in configuration using large-page memory for Linux