CRS-0215: cocould not start resource 'ora. rac3.listener _ rac3.lsnr'

Crs-0215 Rac-VIP add node problem Oracle RAC

Problem:

Netca to createlistener fails with CRS-0215 and the created listener does not start

 

An error crs-0215 is reported when creating a netca experiment for adding nodes

Check Metalink related articles and find that the VIP address is incorrect. Check whether the VIP address of Node 3 is on another node.

[[Email protected] bin] #./crs_stat-T

Name type target State host

------------------------------------------------------------

Ora... b1.inst application online Rac1

Ora... b2.inst application online rac2

Ora. racdb. DB application online rac2

Ora... sm1.asm application online Rac1

Ora... c1.lsnr application online Rac1

Ora. rac1.gsd application online Rac1

Ora. rac1.ons application online Rac1

Ora. rac1.vip application online Rac1

Ora... sm2.asm application online rac2

Ora... c2.lsnr application online rac2

Ora. rac2.gsd application online rac2

Ora. rac2.ons application online rac2

Ora. rac2.vip application online rac2

Ora... c3.lsnr application online offline

Ora. rac3.gsd application online rac3

Ora. rac3.ons application online rac3

Ora. rac3.vip application online Rac1

 

I restarted nodeapps and found that

[[Email protected] bin] #./srvctl stop nodeapps-N rac3

[[Email protected] bin] #./srvctl start nodeapps-N rac3

Rac3: ora. rac3.vip: checkif: Default Gateway is notdefined (host = rac3)

Rac3: ora. rac3.vip: interface eth0 checked failed (host = rac3)

Rac3: ora. rac3.vip: Invalid parameters, or failed tobring up VIP (host = rac3)

Rac3: ora. rac3.vip: checkif: Default Gateway is notdefined (host = rac3)

Rac3: ora. rac3.vip: interface eth0 checked failed (host = rac3)

Rac3: ora. rac3.vip: Invalid parameters, or failed tobring up VIP (host = rac3)

CRS-0215: cocould not start resource 'ora. rac3.listener _ rac3.lsnr '.

 

 

In the crsd. log, the node rac3 failed to start the rac3-vip, and

16:07:11. 548: [crsres] [2751167376] 0 startrunnable: settingcli values

16:07:11. 571: [crsres] [2751167376] 0 attempting to start 'ora. rac3.gsd 'on member 'rac3'

16:07:12. 608: [crsres] [2751167376] 0 start of 'ora. rac3.gsd 'on member 'rac3' succeeded.

16:07:12. 924: [crsres] [2751167376] 0 startrunnable: settingcli values

16:07:12. 942: [crsres] [2751167376] 0 attempting to start 'ora. rac3.vip 'on member 'rac3'

16:07:19. 968: [crsapp] [2751167376] 0 startresource error forora. rac3.vip error code = 1

16:07:23. 470: [crsres] [2751167376] 0 start of 'ora. rac3.vip 'on member 'rac3' failed.

16:07:23. 880: [crsres] [2751167376] 0 attempting to start 'ora. rac3.vip 'on member 'rac1'

16:07:37. 049: [crsres] [2751167376] 0 start of 'ora. rac3.vip 'on member 'rac1' succeeded.

16:07:37. 328: [crsres] [2751167376] 0 startrunnable: settingcli values

16:07:37. 353: [crsres] [2751167376] 0 attempting to start 'ora. rac3.ons 'on member 'rac3'

16:07:39. 397: [crsres] [2751167376] 0 start of 'ora. rac3.ons 'on member 'rac3' succeeded.

16:07:39. 858: [crsres] [2751167376] 0 stopresource: Setting clivalues

16:07:39. 922: [crsres] [2751167376] 0 attempting to stop 'ora. rac3.vip 'on member 'rac1'

16:07:40. 127: [crsres] [2751167376] 0 stop of 'ora. rac3.vip 'onmember 'rac1' succeeded.

16:07:40. 138: [crsres] [2751167376] 0 startrunnable: settingcli values

16:07:40. 138: [crsres] [2751167376] 0 attempting to start 'ora. rac3.vip 'on member 'rac3'

16:07:47. 157: [crsapp] [2751167376] 0 startresource error forora. rac3.vip error code = 1

16:07:50. 315: [crsres] [2751167376] 0 start of 'ora. rac3.vip 'on member 'rac3' failed.

16:07:50. 665: [crsres] [2751167376] 0rac1: CRS-1019: resourceora. rac3.listener _ rac3.lsnr (Application) cannot run on Rac1

 

CRS-1019: Resource ora. rac3.listener _ rac3.lsnr (Application) cannot run on rac2

 

 

16:07:50. 897: [crsres] [2751167376] 0 attempting to start 'ora. rac3.vip 'on member 'rac1'

16:08:03. 033: [crsres] [2751167376] 0 start of 'ora. rac3.vip 'on member 'rac1' succeeded.

 

 

We found that ora. rac3.vip tried both on node rac3 and both failed, and finally restarted on Rac1, resulting in crs-1019

 

Description on Metalink:

Using netca to create listener fails withCRS-0215 and the created listener does not start (document id859835.1)

 

Thisis because the VIP of Node B is not correctly configured, as the listeneris depend on the VIP.

The problem of the VIP is that the nameregistered for public interface in the OCR is changed on the operating systemlevel.

From "oifcfg getif" That theregistered public interface is named "public" command output asfollow:

Public 10.200.0.0 global public
Private-Heartbeat 192.168.1.0 global cluster_interconnect

From ipconofig/ifconfig for node "B" the public interface is named "Public 1" ipconfig command as follow:

Ethernet Adapter public 1:
Connection-specific DNS suffix .:
Description ......: Broadcom bcm5709c netxtreme II gige (ndisvbd client)
Physical address ......: 00-10-18-4f-29-3c
DHCP enabled...
Autoconfiguration enabled...: Yes
IPv4 address ......: 10.200.0.159 (preferred)
Subnet Mask ......: 255.255.255.0
Default Gateway ......: 10.200.0.222
DNS servers ......: 10.200.0.211
NetBIOS over TCPIP ......: Enabled

This means that the VIP for Node B is online onnode a hence you will still not able to start/create the listener on Node B.

Solution

Consultyour system administrator to correct the public interface name on the OS levelfrom "Public 1" to "public"

Then you shoshould restart nodeapps on thatnode B as follow

Srvctl stop nodeapps-N node1
Srvctl start nodeapps-N node1

Then you shoshould see the VIP online on Node B, hence youshocould be able to create the listener.

 

 

This problem occurs due to the VIP issue.

 

The final troubleshooting configuration is correct, so the above solution is not feasible.

 

 

I switched off the VM and changed two NICs.

After the restart, the system returns to normal !!!

[[Email protected] bin] #./srvctl stop nodeapps-N rac3

[[Email protected] bin] #./srvctl start nodeapps-N rac3

[[Email protected] bin] #./crs_stat-T

Name type target State host

------------------------------------------------------------

Ora... b1.inst application online Rac1

Ora... b2.inst application online rac2

Ora. racdb. DB application online rac2

Ora... sm1.asm application online Rac1

Ora... c1.lsnr application online Rac1

Ora. rac1.gsd application online Rac1

Ora. rac1.ons application online Rac1

Ora. rac1.vip application online Rac1

Ora... sm2.asm application online rac2

Ora... c2.lsnr application online rac2

Ora. rac2.gsd application online rac2

Ora. rac2.ons application online rac2

Ora. rac2.vip application online rac2

Ora... c3.lsnr application online rac3

Ora. rac3.gsd application online rac3

Ora. rac3.ons application online rac3

Ora. rac3.vip application online rac3

 

 

 

Feeling: Thank God! Tears flow...

CRS-0215: cocould not start resource 'ora. rac3.listener _ rac3.lsnr'