Processing of the ORACLE 11204 grid after a corrupted CRS disk group

Processing of the ORACLE 11204 grid after a corrupted CRS disk group
The main processing CRS disk group (OCR and Votedisk Disk group) is damaged in case of a CRS disk processing.
1. Analog fault
[Email protected] ~]$ Ocrcheck
Status of Oracle Cluster Registry is as follows:
Version:3
Total Space (Kbytes): 262120
Used Space (Kbytes): 3152
Available Space (Kbytes): 258968
id:1580323731
Device/file Name: +data
Device/file Integrity Check Succeeded

                                Device/File not configured                                Device/File not configured                                Device/File not configured                                Device/File not configured Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged user

[Email protected] ~]$ Sqlplus/as SYSDBA
Sql> select Name,path from V$asm_disk;

NAME PATH

Arch_0000/dev/raw/raw3
Data_0000/dev/raw/raw1
Daa_0000/dev/raw/raw2
Vote2_0000/dev/raw/raw5
/dev/raw/raw4
/dev/raw/raw6

6 rows selected.

Check for OCR backup
[Email protected] grid]$ Ocrconfig-showbackup
Node1 2016/11/01 13:33:12/U01/11.2.0/GRID/CDATA/CLUSTER/BACKUP00.OCR
Node1 2016/11/01 13:33:12/U01/11.2.0/GRID/CDATA/CLUSTER/DAY.OCR
Node1 2016/11/01 13:33:12/U01/11.2.0/GRID/CDATA/CLUSTER/WEEK.OCR
Node1 2016/10/31 17:03:27/U01/11.2.0/GRID/CDATA/CLUSTER/BACKUP_20161031_170327.OCR
Node1 2016/10/31 17:03:12/U01/11.2.0/GRID/CDATA/CLUSTER/BACKUP_20161031_170312.OCR
Node1 2016/10/31 17:02:44/U01/11.2.0/GRID/CDATA/CLUSTER/BACKUP_20161031_170244.OCR

Emptying the CRS disk:
[[email protected] ~]# dd If=/dev/zero of=/dev/raw/raw1 bs=10m count=100
100+0 Records in
100+0 Records out
1048576000 bytes (1.0 GB) copied, 15.8185 s, 66.3 MB/s

2. Stop the cluster:
[[email protected] grid]# Crsctl stop CRS
crs-2791:starting shutdown of Oracle high availability services-managed resources on ' Node1 '
Crs-2673:attempting to stop ' ora.crsd ' on ' node1 '
crs-2790:starting shutdown of Cluster ready services-managed resources on ' Node1 '
crs-2673:attempting to stop ' Ora. Listener_scan1.lsnr ' on ' node1 '
Crs-2673:attempting to stop ' ora.node2.vip ' on ' node1 '
crs-2673:attempting to stop ' Ora. Arch.dg ' on ' node1 '
crs-2673:attempting to stop ' Ora. Data.dg ' on ' node1 '
crs-2673:attempting to stop ' Ora. Vote2.dg ' on ' node1 '
Crs-2673:attempting to stop ' ora.registry.acfs ' on ' node1 '
Crs-2673:attempting to stop ' ora.orcl.db ' on ' node1 '
crs-2673:attempting to stop ' Ora. Listener.lsnr ' on ' node1 '
Crs-2673:attempting to stop ' ORA.CVU ' on ' node1 '
Crs-2673:attempting to stop ' ora.oc4j ' on ' node1 '
Crs-2677:stop of ' ora.node2.vip ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.cvu ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Listener.lsnr ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.node1.vip ' on ' node1 '
Crs-2677:stop of ' Ora. Listener_scan1.lsnr ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.scan1.vip ' on ' node1 '
Crs-2677:stop of ' ora.orcl.db ' on ' Node1 ' succeeded
crs-2673:attempting to stop ' Ora. Daa.dg ' on ' node1 '
Crs-2677:stop of ' ora.node1.vip ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.scan1.vip ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Daa.dg ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Arch.dg ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Vote2.dg ' on ' Node1 ' succeeded
crs-4549:unexpected Disconnect while executing shutdown request.
Crs-5022:stop of resource "ORA.CRSD" failed:current State is "intermediate"
Crs-2675:stop of ' ora.crsd ' on ' Node1 ' failed
crs-2679:attempting to "Ora.crsd ' on ' node1 '
Crs-2681:clean of ' ora.crsd ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ORA.CRF ' on ' node1 '
Crs-2673:attempting to stop ' ora.ctssd ' on ' node1 '
Crs-2673:attempting to stop ' ora.evmd ' on ' node1 '
Crs-2673:attempting to stop ' ora.asm ' on ' node1 '
Crs-2673:attempting to stop ' ora.mdnsd ' on ' node1 '
Crs-2673:attempting to stop ' ora.drivers.acfs ' on ' node1 '
Crs-2677:stop of ' ORA.CRF ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.mdnsd ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.evmd ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.ctssd ' on ' Node1 ' succeeded
Crs-5022:stop of resource "ora.asm" failed:current State is "UNKNOWN"
Crs-2675:stop of ' ora.asm ' on ' Node1 ' failed
crs-2679:attempting to "Ora.asm ' on ' node1 '

3. Start CRS, repair OCR and vote
Start CRS at Node1
[Email protected] grid]# crsctl start Crs-excl-nocrs
Crs-4123:oracle High Availability Services have been started.
Crs-2672:attempting to start ' ora.mdnsd ' on ' node1 '
Crs-2676:start of ' ora.mdnsd ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.gpnpd ' on ' node1 '
Crs-2676:start of ' ora.gpnpd ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.cssdmonitor ' on ' node1 '
Crs-2672:attempting to start ' ora.gipcd ' on ' node1 '
Crs-2676:start of ' ora.cssdmonitor ' on ' Node1 ' succeeded
Crs-2676:start of ' ora.gipcd ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.cssd ' on ' node1 '
Crs-2672:attempting to start ' Ora.diskmon ' on ' node1 '
Crs-2676:start of ' Ora.diskmon ' on ' Node1 ' succeeded
Crs-2676:start of ' ora.cssd ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.drivers.acfs ' on ' node1 '
crs-2679:attempting to "Ora.cluster_interconnect.haip ' on ' node1 '
Crs-2672:attempting to start ' ora.ctssd ' on ' node1 '
Crs-2681:clean of ' ora.cluster_interconnect.haip ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.cluster_interconnect.haip ' on ' node1 '
Crs-2676:start of ' ora.drivers.acfs ' on ' Node1 ' succeeded
Crs-2676:start of ' ora.ctssd ' on ' Node1 ' succeeded
Crs-2676:start of ' ora.cluster_interconnect.haip ' on ' Node1 ' succeeded
Crs-2672:attempting to start ' ora.asm ' on ' node1 '
Crs-2676:start of ' ora.asm ' on ' Node1 ' succeeded

[Email protected] grid]# Ps-ef|grep Smon
Grid 7264 1 0 15:34? 00:00:00 asmSmon+ASM1
Root 7317 6509 0 15:34 pts/0 00:00:00 grep smon

Creating ASM Disk Groups for OCR and Votedisk
[Email protected] ~]$ Sqlplus/as sysasm
sql> Create DiskGroup Data external redundancy disk '/dev/raw/raw1 ' attribute ' COMPATIBLE. ASM ' = ' 11.2 ';

Restore OCR
[Email protected] cluster]# Ocrconfig-restore WEEK.OCR
Recover vote Disk
Attention:
1. If you are using Crsctl replace Votedisk +diskgroup in ASM to complete the repair
2, if in the cluster file system, then need to view votedisk information, crstl query CSS Votedisk, and then delete the original invalid Votedisk, and then add Votedisk
Command: Crsctl add CSS votedisk ' path '
In this example, ASM is used, so the following method is used to recover.
[Email protected] cluster]# crsctl replace Votedisk +data
Successful addition of voting disk 4414805F23EE4FCBBFC5B63EA5A121B5.
Successfully replaced voting disk group with +data.
crs-4266:voting file (s) successfully replaced

Close CRS
[Email protected] cluster]# Crsctl stop crs-f
crs-2791:starting shutdown of Oracle high availability services-managed resources on ' Node1 '
Crs-2673:attempting to stop ' ora.crsd ' on ' node1 '
crs-2790:starting shutdown of Cluster ready services-managed resources on ' Node1 '
Crs-2673:attempting to stop ' ora.node2.vip ' on ' node1 '
Crs-2673:attempting to stop ' ORA.CVU ' on ' node1 '
Crs-2677:stop of ' ora.cvu ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.node2.vip ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.oc4j ' on ' node1 '
Crs-2677:stop of ' ora.oc4j ' on ' Node1 ' succeeded
crs-2673:attempting to stop ' Ora. Listener_scan1.lsnr ' on ' node1 '
crs-2673:attempting to stop ' Ora. Listener.lsnr ' on ' node1 '
Crs-2677:stop of ' Ora. Listener_scan1.lsnr ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.scan1.vip ' on ' node1 '
Crs-2677:stop of ' Ora. Listener.lsnr ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.node1.vip ' on ' node1 '
Crs-2677:stop of ' ora.scan1.vip ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.node1.vip ' on ' Node1 ' succeeded
crs-2673:attempting to stop ' Ora. Arch.dg ' on ' node1 '
crs-2673:attempting to stop ' Ora. Data.dg ' on ' node1 '
crs-2673:attempting to stop ' Ora. Vote2.dg ' on ' node1 '
Crs-2673:attempting to stop ' ora.registry.acfs ' on ' node1 '
Crs-2673:attempting to stop ' ora.orcl.db ' on ' node1 '
Crs-2677:stop of ' Ora. Arch.dg ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Vote2.dg ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.orcl.db ' on ' Node1 ' succeeded
crs-2673:attempting to stop ' Ora. Daa.dg ' on ' node1 '
Crs-2677:stop of ' ora.registry.acfs ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Daa.dg ' on ' Node1 ' succeeded
Crs-2677:stop of ' Ora. Data.dg ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.asm ' on ' node1 '
Crs-2677:stop of ' ora.asm ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.ons ' on ' node1 '
Crs-2677:stop of ' ora.ons ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.net1.network ' on ' node1 '
Crs-2677:stop of ' ora.net1.network ' on ' Node1 ' succeeded
Crs-2792:shutdown of Cluster Ready services-managed resources on ' Node1 ' have completed
Crs-2677:stop of ' ora.crsd ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.mdnsd ' on ' node1 '
Crs-2673:attempting to stop ' ora.drivers.acfs ' on ' node1 '
Crs-2673:attempting to stop ' ORA.CRF ' on ' node1 '
Crs-2673:attempting to stop ' ora.ctssd ' on ' node1 '
Crs-2673:attempting to stop ' ora.evmd ' on ' node1 '
Crs-2673:attempting to stop ' ora.asm ' on ' node1 '
Crs-2677:stop of ' ora.mdnsd ' on ' Node1 ' succeeded
Crs-2677:stop of ' ORA.CRF ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.evmd ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.ctssd ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.asm ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.cluster_interconnect.haip ' on ' node1 '
Crs-2677:stop of ' ora.cluster_interconnect.haip ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.cssd ' on ' node1 '
Crs-2677:stop of ' ora.drivers.acfs ' on ' Node1 ' succeeded
Crs-2677:stop of ' ora.cssd ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.gipcd ' on ' node1 '
Crs-2677:stop of ' ora.gipcd ' on ' Node1 ' succeeded
Crs-2673:attempting to stop ' ora.gpnpd ' on ' node1 '
Crs-2677:stop of ' ora.gpnpd ' on ' Node1 ' succeeded
Crs-2793:shutdown of Oracle High availability services-managed resources on ' Node1 ' have completed
Crs-4133:oracle High Availability Services have been stopped.

Start CRS
[[email protected] cluster]# Crsctl start CRS
Crs-4123:oracle High Availability Services have been started.

Processing of the ORACLE 11204 grid after a corrupted CRS disk group