Oracle 10g RAC root. Sh failure at final check of Oracle CRS stack 10 Solution

 

I. Problem Description

 

Installation environment: Oracle Linux 6.1

Database: 10.2.0.1

 

When Oracle 10g RAC is installed, an error is reported when root. Sh is executed on the first node, as shown below:

 

[Root @ Rac1 ~] #/U01/APP/10.2.0/GRID/root. Sh

Warning: directory '/u01/APP/10.2.0' is notowned by root

Warning: directory '/u01/app' is not ownedby Root

Warning: directory '/u01' is not owned byroot

Checking to see if Oracle CRS stack isalready configured

 

Setting the permissions on OCR backupdirectory

Setting up NS Directories

Oracle cluster registry configurationupgraded successfully

Warning: directory '/u01/APP/10.2.0' is notowned by root

Warning: directory '/u01/app' is not ownedby Root

Warning: directory '/u01' is not owned byroot

Successfully accumulated necessary ocrkeys.

Using ports: CSS = 49895 CRS = 49896 evmc = 49898and evmr = 49897.

Node <nodenumber>: <nodename> <private interconnect Name>

Node 1: Rac1 rac1-priv Rac1

Node 2: rac2 rac2-priv rac2

Creating OCR keys for user 'root', privgrp 'root '..

Operation successful.

Now formatting voting device:/dev/raw/raw3

 

Now formatting voting device:/dev/raw/raw4

Now formatting voting device:/dev/raw/raw5

Format of 3 voting devices complete.

Startup will be queued to init within 90seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within600 seconds.

 

Failure at final check of Oracle CRS stack.

10

 

II. This issue is described in the mos document:

 

2.1 document 1:

Root. Sh failed at failure at final check oforacle CRS stack 10 [ID 725878.1]

 

Case

This particle case is caused by the osinit system does not working.

"Failure at final check of Oracle CRS stack.
10"
Means CRS daemon did not startup During 600 seconds period.

 

In the root. Sh script, it adds CRS relatedentry in/etc/inittab, Run "init Q" and keep CT 3 CRS related daemonprocesses to start, eg:

Init.css d
Init. crsd
Init. evmd

 

With init system problem, None of thesedaemon processes are spawned, this causes CRS process startup failure as theyrely on the CRS daemon processes to start first.
-- The init system problem is faulty, and the process cannot be started. You can verify this problem by using the following methods:

This can be verified by adding a simple entry in/etc/inittab:

Test: 2: Once:/usr/bin/echo "hellotest">/tmp/test. Log

Run "init Q" as root user. If the init is working, then there shouldbe a file/tmp/test. Log generated.

 

Solution

 

-- Only the solution on AIX is provided on Mos, as shown below:

Please consult with System Administrator tofix initissue.

Here the solution is only valid for aixplatform:

1. Starting the script install_assist (aixgui utility installation compliance)
2. Updating for example the date, then exit install_assist properly
3. reboot the system
After that daemon process in/etc/inittab started, CRS installation completed.

 

 

2.2 document 2:

Clusterware fails to start duringroot. Sh -- "failure at final check of Oracle CRS stack 10" [id329450.1]

 

The Oracle clusterware runs as root, but for some operations itneed to run as the Oracle user, and uses the "Su-L" which invokesthe Oracle User Shell login/profile script. if that shell profile script hasInteractiveOrCPU boundOperations or prompts this may affect theclusterware operation.

 

-- In this document, parameters in. bash_profile have interactive parameters. You can delete these parameters.

 

Other documents:

Troubleshooting 10g or 11.1 oracleclusterware root. Sh problems [ID 240001.1]

 

 

Iii. Problem Analysis

 

View related logs:

[Oracle @ Rac1 client] $ pwd

/U01/APP/10.2.0/GRID/log/Rac1/client

[Oracle @ Rac1 client] $ ls

Clscfg_6337.log CLSC. Log CSS. Log ocrconfig_6285.log

 

[Oracle @ Rac1 client] $ tail-30 CSS. Log

23:23:15. 565: [cssclnt] [3681171200] clsssinitnative: connect failed, RC 9

23:23:16. 977: [cssclnt] [3681171200] clsssinitnative: connect failed, RC 9

23:23:18. 390: [cssclnt] [3681171200] clsssinitnative: connect failed, RC 9

23:23:19. 885: [cssclnt] [3681171200] clsssinitnative: connect failed, RC 9

 

[Oracle @ Rac1 client] $ tail-10 CLSC. Log

Oracle Database 10g CRS release 10.2.0.1.0production copyright 1996,200 5 oracle. All rights reserved.

23:24:51. 389: [Default] [4093163264] terminating clsd session

23:25:00. 274: [Default] [135894784] terminating clsd session

 

[Oracle @ Rac1 client] $ tail clscfg_6337.log

Oracle Database 10g CRS release 10.2.0.1.0production copyright 1996,200 5 oracle. All rights reserved.

23:12:24. 477: [clscfg] [1566725888] clscfg: nodelist is [rac1rac2]

 

[Oracle @ Rac1 Rac1] $ cat alertrac1.log

2012-07-12 10:06:10. 703

[Client (6285)] CRS-1006: the OCR location/dev/raw/raw2 is inaccessible. Details in/u01/APP/10.2.0/GRID/log/Rac1/client/ocrconfig_6285.log.

2012-07-12 10:06:11. 076

[Client (6285)] CRS-1001: the OCR wasformatted using version 2.

2012-07-12 10:12:24. 479

[Client (6337)] CRS-1801: Cluster crsconfigured with nodes Rac1 rac2.

 

 

-- Run the following command on the root node to clear the OCR information:

[Root @ Rac1 ~] # Sh/u01/APP/10.2.0/GRID/install/rootdeinstall. Sh

Removing contents from OCR mirror Device

2560 + 0 records in

2560 + 0 records out

10485760 bytes (10 MB) Copied, 2.46509 S, 4.3 MB/S

Removing contents from OCR Device

2560 + 0 records in

2560 + 0 records out

10485760 bytes (10 MB) Copied, 1.18886 S, 8.8 MB/S

 

 

Then the problem still persists when running root. Sh.

 

 

The following method is used:

1. Disable Firewall

I have disabled the firewall before installation, so I will check it here.

 

[Root @ Rac1 TMP] # service iptables status

Iptables: Firewall is not running.

[Root @ Rac1 TMP] # chkconfig iptables -- list

Iptables 0: off 1: off 2: off 3: off 4: off 5: off 6: Off

 

2. commented out the following file:

[Root @ Rac1 TMP] # Cat/etc/PAM. d/Other

# % PAM-1.0

Auth required pam_deny.so

Account required pam_deny.so

Password required pam_deny.so

Session required pam_deny.so

 

3. Delete related socket:

# Rm-F/usr/tmp/. Oracle /*

# Rm-F/tmp/. Oracle /*

# Rm-F/var/tmp/. Oracle /*

 

Unable to connect to cluster ManagerOra-29701 as network socket files are removed [ID 391790.1]

 

Run sh/u01/APP/10.2.0/GRID/install/rootdeinstall. SH and install it again. The problem persists and may be caused by compatibility issues.

 

Later, I switched OS to RedHat 5.4 and installed it successfully. It may be due to Oracle 10 Gb compatibility issues on Oracle Linux 6. On Oracle Linux 6, I tested Oracle 11.2.0.3 RAC, installation is correct.

 

 

 

 

 

 

Bytes -------------------------------------------------------------------------------------------------------

All rights reserved. reprinted articles are allowed, but source addresses must be indicated by links. Otherwise, the documents will be held legally responsible!

Skype: tianlesoftware

QQ: tianlesoftware@gmail.com

Email: tianlesoftware@gmail.com

Blog: http://www.tianlesoftware.com

WEAVER: http://weibo.com/tianlesoftware

Twitter: http://twitter.com/tianlesoftware

Facebook: http://www.facebook.com/tianlesoftware

LinkedIn: http://cn.linkedin.com/in/tianlesoftware

 

 

------- Add a group to describe the relationship between Oracle tablespace and data files in the remarks section. Otherwise, reject the application ----

Dba1 group: 62697716 (full); dba2 group: 62697977 (full) dba3 group: 62697850 (full)

Super DBA group: 63306533 (full); dba4 group: 83829929 dba5 group: 142216823

Dba6 group: 158654907 dba7 group: 172855474 DBA group: 104207940