Oracle 1204 RAC failover test (III)

After two tests, RAC is very fragile.
1. Remove the public network cable of Rac1 and stand next to rac2 to see the changes. It is found that the VIP is quickly converted to rac2 and users can still use it.
2.1 minutes later, rac2 restarts automatically. Check that the shared disk cannot be mounted. Another colleague is in config San and cannot determine whether the shared disk is actually faulty.
3. Simply perform a more aggressive test, remove the power of the two databases, plug it back, and restart the system. It is found that CRS cannot be started.
[Root @ racdb02 install] # crs_stat-T
CRS-0184: cannot communicate with the CRS daemon.
4. I have found many methods on the network, but they have no effect. There is no proper method on Metalink.
5. Run root102.sh on the two nodes respectively.
[Oracle @ racdb01 ~] $/U01/Oracle/product/10.2/crs1/install/root102.sh
6. Reboot two nodes
7. [root @ racdb01 Oracle] # crs_stat-T
Name type target State host
------------------------------------------------------------
Ora. RAC. DB application online unknown racdb02
Ora... s1.inst application online unknown racdb01
Ora... s2.inst application online unknown racdb02
Ora... esdb. CS application online offline
Ora... es1.srv application online offline
Ora... es2.srv application online offline
Ora... 01. lsnr application online racdb01
Ora... b01.gsd application online racdb01
Ora... b01.ons application online racdb01
Ora... b01.vip application online racdb01
Ora... 02. lsnr application online racdb02
Ora... b02.gsd application online unknown racdb02
Ora... b02.ons application online unknown racdb02
Ora... b02.vip application online racdb02
8. Failed to delete instance, failed to delete service racdb, and failed to delete listener
9. crs_start-all
The service cannot be started.
10. Uninstall CRS, reinstall and upgrade to 1204.
11. All CRS services are started normally.
12. srvctl add database-D Rac-O/u01/Oracle/product/10.2/db1/
13. srvctl add instance-D Rac-I Rac1-N racdb01
Srvctl add instance-D Rac-I rac2-N racdb02
14. Reboot on two nodes
15. It is found that the startup of rac2 is normal, but the instance of Rac1 cannot be started.
[Oracle @ racdb01 ~] $ Crs_stat-T
Name type target State host
------------------------------------------------------------
Ora. RAC. DB application online racdb02
Ora... s1.inst application offline
Ora... s2.inst application online racdb02
Ora... 01. lsnr application online racdb01
Ora... b01.gsd application online racdb01
Ora... b01.ons application online racdb01
Ora... b01.vip application online racdb01
Ora... 02. lsnr application online racdb02
Ora... b02.gsd application online racdb02
Ora... b02.ons application online racdb02
Ora... b02.vip application online racdb02
16. srvctl remove instance-D Rac-I Rac1
17. srvctl add instance-D Rac-I Rac1-N racdb01
18. Try to start the Rac1 Service
[Oracle @ racdb01 ~] $ Srvctl start instance-D Rac-I Rac1-O Mount;
PRKP-1001: Error starting instance maid on node racdb01
CRS-1028: dependency analysis failed because:
CRS-0223: Resource 'ora. RAC. rac1.inst 'has placement error.

[Oracle @ racdb01 ~] $ Crs_start ora. RAC. rac1.inst
Attempting to start 'ora. RAC. rac1.inst 'on member 'racdb01'
'Ora. RAC. rac1.inst 'on member 'racdb01' has experienced an unrecoverable failure.
Human intervention required to resume its availability.
CRS-0215: cocould not start resource 'ora. RAC. rac1.inst '.

[Oracle @ racdb01 admin] $ crs_start ora. RAC. rac1.inst
CRS-1028: dependency analysis failed because:
'Resource in unknown state: ora. RAC. rac1.inst'
 
CRS-0223: Resource 'ora. RAC. rac1.inst 'has placement error.

19. I continued to search for information on the Internet. After a long time, it was still invalid.
20. I saw an article on Metalink saying that tnsnarac. ora is faulty.
21. Check my tnsnarac. ora and find that the original racdb service (for transparent failover) is still in use, but it is not set after the CRS is re-installed. Delete it
22. [Oracle @ racdb01 admin] $ crs_start ora. RAC. rac1.inst
Attempting to start 'ora. RAC. rac1.inst 'on member 'racdb01'
Start of 'ora. RAC. rac1.inst 'on member 'racdb01' succeeded.
23. [Oracle @ racdb01 admin] $ crs_stat-T
Name type target State host
------------------------------------------------------------
Ora. RAC. DB application online racdb02
Ora... s1.inst application online racdb01
Ora... s2.inst application online racdb02
Ora... 01. lsnr application online racdb01
Ora... b01.gsd application online racdb01
Ora... b01.ons application online racdb01
Ora... b01.vip application online racdb01
Ora... 02. lsnr application online racdb02
Ora... b02.gsd application online racdb02
Ora... b02.ons application online racdb02
Ora... b02.vip application online racdb02
Finally, I started it. I applaud it for two days!