環境介紹:
雙機作業系統:solaris 10資料庫版本:oracle 11g R1 64bit當前主機的主機名稱和IP:fly-db2 192.168.20.21當前備機的主機名稱和IP:fly-db1 192.168.20.20
背景:
主備機的實體記憶體均為94G,近期對備機的實體記憶體擴容一倍,並修改了資料庫層面的相關記憶體參數,以及主備機的作業系統參數shmsys:shminfo_shmmax,注意是主備的shmsys:shminfo_shmmax參數均已修改,詳情請查看下面連結的博文
http://fly1116.blog.51cto.com/8301004/1343788
今晚對fly-db1主機目前的備機)的實體記憶體擴容一倍後,重啟作業系統,因主備機的shmsys:shminfo_shmmax參數均已經修改,重啟作業系統後,理論上該參數設定生效,通過雙機軟體將資料庫從fly-db2切換到fly-db1,切換失敗,在雙機軟體中,查看到out of memory錯誤,在資料庫alert日誌,查看到如下報錯:
WARNING: The system does not seem to be configuredoptimally. Creating a segment of size 0x0000000220000000failed. Please change the shm parameters so thata segment can be created for this size. While this isnot a fatal issue, creating one segment may improve performance如果手工啟動,則報錯如下:SQL> startupORA-27102: out of memorySVR4 Error: 22: Invalid argumentbash-3.00$ oerr ora 2710227102, 00000, "out of memory"// *Cause: Out of memory// *Action: Consult the trace file for details
從以上報錯分析,因為使用的是同一個資料庫,作業系統層面只修改了shmsys:shminfo_shmmax參數,定位為當前備機fly-db1該參數設定未生效
1、對比主備機的/etc/system檔案,是一致的
bash-3.00#hostnamefly-db1bash-3.00# diff /etc/system /home/oracle/system_flydb2_131220.bak
2、百度之solaris out of memory,查看到以下blog,文章中涉及/etc/project檔案,我們可以這麼理解/etc/project檔案,如果/etc/system和/etc/project中設定了相同的參數,則作業系統會忽略/etc/system的設定,而採用/etc/project的設定,/etc/project檔案可以針對特定使用者佈建只對自己生效的參數
https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixing Oracle on Solaris 10 : Fixing the 'ORA-27102: out of memory' Error
3、查看到當前備機fly-db1主機的/etc/project中設定了max-shm-memory的最大大小為60G,即最大的共用記憶體段為60G,如下所示:
bash-3.00# hostnamefly-db1bash-3.00# cat /etc/projectsystem:0::::user.root:1::::noproject:2::::default:3::::project.max-shm-memory=(privileged,64424509440,deny)group.staff:10::::bash-3.00# bcscale=564424509440/1024/1024/102460.00000
4、查看當前主機fly-db2)的/etc/project內容,未設定max-shm-memory的最大值,所以在該作業系統,生效的是/etc/system的設定
bash-3.00#hostnamefly-db2bash-3.00# cat /etc/projectsystem:0::::user.root:1::::noproject:2::::default:3::::group.staff:10::::
5、將當前備機fly-db1的/etc/project修改成和當前主機的/etc/project一樣
bash-3.00#hostnamefly-db1bash-3.00# cat /etc/projectsystem:0::::user.root:1::::noproject:2::::default:3::::group.staff:10::::
6、重啟作業系統
bash-3.00# syncbash-3.00# syncbash-3.00# syncbash-3.00# shutdown -i6 -g0 -y
7、雙機切換,成功切換到備機fly-db1
以下內容來自:https://blogs.oracle.com/mandalika/entry/oracle_on_solaris_10_fixingSymptom:As part of a database tuning effort you increase the SGA/PGA sizes; and Oracle greets with an ORA-27102:out of memory error message. The system had enough free memory to serve the needs of Oracle.SQL> startupORA-27102: out of memorySVR4 Error: 22: Invalid argumentDiagnosis$ oerr ORA 2710227102, 00000, "out of memory"// \*Cause: Out of memory// \*Action: Consult the trace file for detailsNot so helpful. Let's look the alert log for some clues.% tail -2 alert.logWARNING: EINVAL creating segment of size 0x000000028a006000fix shm parameters in /etc/system or equivalentOracle is trying to create a 10G shared memory segment (depends on SGA/PGA sizes), but operating system (Solaris in this example)responded with an invalid argument (EINVAL) error message. There is a little hint about setting shm parameters in /etc/system.Prior to Solaris 10, shmsys:shminfo_shmmax parameter has to be set in /etc/system with maximum memory segment value that canbe created. 8M is the default value on Solaris 9 and prior versions; where as 1/4th of the physical memory is the default on Solaris 10and later. On a Solaris 10 (or later) system, it can be verified as shown below:% prtconf | grep MemMemory size: 32760 Megabytes% id -puid=59008(oracle) gid=10001(dba) projid=3(default)% prctl -n project.max-shm-memory -i project 3project: 3: defaultNAME PRIVILEGE VALUE FLAG ACTION RECIPIENTproject.max-shm-memoryprivileged 7.84GB - deny -system 16.0EB max deny -Now it is clear that the system is using the default value of 8G in this scenario, where as the application (Oracle) is trying tocreate a memory segment (10G) larger than 8G. Hence the failure.So, the solution is to configure the system with a value largeenough for the shared segment being created, so Oracle succeeds in starting up the database instance.On Solaris 9 and prior releases, it can be done by adding the following line to /etc/system, followed by a reboot for the systemto pick up the new value.set shminfo_shmmax = 0x000000028a006000However shminfo_shmmax parameter was obsoleted with the release of Solaris 10; and Sun doesn't recommend setting this parameterin /etc/system even though it works as expected.On Solaris 10 and later, this value can be changed dynamically on a per project basis with the help of resource control facilities . This ishow we do it on Solaris 10 and later:% prctl -n project.max-shm-memory -r -v 10G -i project 3% prctl -n project.max-shm-memory -i project 3project: 3: defaultNAME PRIVILEGE VALUE FLAG ACTION RECIPIENTproject.max-shm-memoryprivileged 10.0GB - deny -system 16.0EB max deny -Note that changes made with the prctl command on a running system are temporary, and will be lost when the system is rebooted. To makethe changes permanent, create a project with projadd command and associate it with the user account as shown below:% projadd -p 3 -c 'eBS benchmark' -U oracle -G dba -K 'project.max-shm-memory=(privileged,10G,deny)' OASB% usermod -K project=OASB oracleFinally make sure the project is created with projects -l or cat /etc/project commands.% projects -l......OASBprojid : 3comment: "eBS benchmark"users : oraclegroups : dbaattribs: project.max-shm-memory=(privileged,10737418240,deny)% cat /etc/project......OASB:3:eBS benchmark:oracle:dba:project.max-shm-memory=(privileged,10737418240,deny)With these changes, Oracle would start the database up normally.SQL> startupORACLE instance started.Total System Global Area 1.0905E+10 bytesFixed Size 1316080 bytesVariable Size 4429966096 bytesDatabase Buffers 6442450944 bytesRedo Buffers 31457280 bytesDatabase mounted.Database opened.
本文出自 “斜陽悠悠寸草心” 部落格,請務必保留此出處http://fly1116.blog.51cto.com/8301004/1343864