Testing the importance of OLR in Oracle 11g cluster

Testing the importance of OLR in Oracle 11g cluster

Called an Oracle local Registry (OLR): All node in a cluster have a Local Registry for node-specific Resources

Test one: Simulate the OLR abnormal loss situation:

Here first will OLR Renam
[Email protected] cdata]# MV VMRAC2.OLR Vmrac2.olr.bak

Then try to start the CRS
[[email protected] cdata]# Crsctl start CRS
Crs-4124:oracle High Availability Services startup failed.
Crs-4000:command Start failed, or completed with errors.

We then observe the log output of the cluster alert log:

[Email protected] vmrac2]$ TAILF Alertvmrac2.log
[OHASD (2495)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage Operating System Error [No such file or directory] [2]]. Details at (: OHAS00106:) in

/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
2014-06-16 16:51:59.491
[OHASD (2506)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage Operating System Error [No such file or directory] [2]]. Details at (: OHAS00106:) in

/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
2014-06-16 16:51:59.698
[OHASD (2517)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage Operating System Error [No such file or directory] [2]]. Details at (: OHAS00106:) in

/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
2014-06-16 16:51:59.901
[OHASD (2528)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage Operating System Error [No such file or directory] [2]]. Details at (: OHAS00106:) in

/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
2014-06-16 16:52:00.113
[OHASD (2539)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage Operating System Error [No such file or directory] [2]]. Details at (: OHAS00106:) in

/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
[Client (2554)] Crs-10001:crs-10132:no msg for has:crs-10132 [10][60]
2014-06-16 16:56:00.720
[OHASD (2717)] Crs-2112:the OLR Service started on node Vmrac2.
2014-06-16 16:56:00.788
[OHASD (2717)] Crs-1301:oracle High Availability Service started on node Vmrac2.
2014-06-16 16:56:00.855
[OHASD (2717)] Crs-8017:location:/etc/oracle/lastgasp has 2 reboot advisory log files, 0 were announced and 0 errors

Occurred
2014-06-16 16:56:01.836
[/u02/app/11.2.0.3/grid/bin/orarootagent.bin (2768)] Crs-5016:process "/u02/app/11.2.0.3/grid/bin/acfsload" spawned by agent

"/u02/app/11.2.0.3/grid/bin/orarootagent.bin" for Action "check" Failed:details at "(: CLSN00010:)" in

"/u02/app/11.2.0.3/grid/log/vmrac2/agent/ohasd/orarootagent_root/orarootagent_root.log"
2014-06-16 16:56:19.876
[OHASD (2717)]crs-2302:cannot get GPNP profile.Error Clsgpnp_no_daemon (GPNPD DAEMON is not running).
2014-06-16 16:56:19.909
[GPNPD (2873)] CRS-2328:GPNPD started on node Vmrac2.
2014-06-16 16:56:22.751
[CSSD (2947)] CRS-1713:CSSD Daemon is started in clustered mode
2014-06-16 16:56:24.073
[OHASD (2717)] Crs-2767:resource State recovery not attempted for ' Ora.diskmon ' as it target State is OFFLINE
2014-06-16 16:56:32.512
[CSSD (2947)] Crs-1707:lease Acquisition for node VMRAC2 number 2 completed
2014-06-16 16:56:33.798
[CSSD (2947)] CRS-1605:CSSD voting file is online:ORCL:CRSVOL1; Details In/u02/app/11.2.0.3/grid/log/vmrac2/cssd/ocssd.log.
2014-06-16 16:56:40.342
[CSSD (2947)] CRS-1601:CSSD reconfiguration complete. Active nodes is Vmrac1 vmrac2.
2014-06-16 16:56:42.635
[CTSSD (3009)] Crs-2401:the Cluster Time synchronization Service started on host VMRAC2.
2014-06-16 16:56:42.635
[CTSSD (3009)] Crs-2407:the new Cluster time synchronization Service reference node is host VMRAC1.
2014-06-16 16:56:46.726
[CTSSD (3009)] Crs-2408:the clock on host VMRAC2 have been updated by the Cluster time synchronization Service to be

Synchronous with the mean cluster time.
[Client (3047)] CRS-10001:16-JUN-14 16:56 acfs-9391:checking for existing ADVM/ACFS installation.
[Client (3060)] crs-10001:16-jun-14 16:56 acfs-9392:validating ADVM/ACFS installation files for operating system.
[Client (3062)] crs-10001:16-jun-14 16:56 acfs-9393:verifying ASM Administrator setup.
[Client (3065)] CRS-10001:16-JUN-14 16:56 acfs-9308:loading installed ADVM/ACFS drivers.
[Client (3069)] crs-10001:16-jun-14 16:56 acfs-9154:loading ' Oracleoks.ko ' driver.
[Client (3080)] crs-10001:16-jun-14 16:56 acfs-9154:loading ' Oracleadvm.ko ' driver.
[Client (3096)] crs-10001:16-jun-14 16:56 acfs-9154:loading ' Oracleacfs.ko ' driver.
[Client (3180)] crs-10001:16-jun-14 16:56 acfs-9327:verifying ADVM/ACFS devices.
[Client (3183)] CRS-10001:16-JUN-14 16:56 acfs-9156:detecting control device '/dev/asm/.asm_ctl_spec '.
[Client (3187)] CRS-10001:16-JUN-14 16:56 acfs-9156:detecting control device '/dev/ofsctl '.
[Client (3193)] Crs-10001:16-jun-14 16:56 acfs-9322:completed

Test two: Empty the contents of the OLR, using an empty file instead:

Observe the Alert.log content as follows:

[OHASD (5451)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

accessing the physical storage]. Details at (: OHAS00106:) In/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
2014-06-16 17:19:02.723
[OHASD (5462)] Crs-0704:oracle High Availability Service aborted due to Oracle Local Registry error [Procl-26:error while

Accessing the physical storage]. Details at (: OHAS00106:) In/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log.
[Client (5477)] Crs-10001:crs-10132:no msg for has:crs-10132 [10][60]

Observe the contents of the corresponding Ohasd.log log:

[Email protected] vmrac2]$ Tail-300/u02/app/11.2.0.3/grid/log/vmrac2/ohasd/ohasd.log
2014-06-16 17:19:02.722: [ocrosd][1923920288]utread:3: Problem reading buffer 150c4000 Buflen 4096 retval 0 Phy_offset

102400 Retry 5
2014-06-16 17:19:02.722: [ocrraw][1923920288]propriogid:1_1:failed to read the whole bootblock. assumes invalid format.
2014-06-16 17:19:02.722: [Ocrraw][1923920288]proprioini:all disks is not OCR/OLR formatted
2014-06-16 17:19:02.722: [ocrraw][1923920288]proprinit:could not open raw device
2014-06-16 17:19:02.722: [ocrapi][1923920288]a_init:16!: Backend init unsuccessful: [26]
2014-06-16 17:19:02.723: [crsocr][1923920288] OCR context init failure. Error:procl-26:error while accessing the

Physical storage
2014-06-16 17:19:02.723: [default][1923920288] Created alert: (: OHAS00106:): OLR initialization failed, error:procl-

26:error while accessing the physical storage
2014-06-16 17:19:02.723: [default][1923920288][panic] OHASD exiting; Could not init OLR
2014-06-16 17:19:02.723: [default][1923920288] Done

Summarize:
Based on the above test, you can find that OHASD (Oracle High Availability Service) relies on configuration information in OLR (Oracle Local Registry) if OLR

Often, or missing, causes the OHASD process to start failing.