Analysis of crs-4000 errors in oracle11.2.0.4rac setup

On a server whose system environment is rhel6.5, the two node RAC with the database version oracle11.2.0.4 is set up. The following error occurs when installing the GRID: (Note: vote, data, and flashback are on the storage device) ASMcreatedandstartedsuccessfully. diskGroupVOTEmountedsuccessfully. clscfg:-installmode

On a server whose system environment is rhel6.5, the following error occurs when the database version is oracle11.2.0.4 and RAC is set up. (Note: vote, data, and Flash back are all stored on the storage device.) ASM created and started successfully. disk Group VOTE mounted successfully. clscfg:-install mode

On a server whose system environment is rhel6.5, the following error occurs when the database version is oracle11.2.0.4 and RAC is set up:

(Note: the vote, data, and Flash back disks are stored on the storage device)

ASM created and started successfully.

Disk Group VOTE mounted successfully.

Clscfg:-install mode specified

Clscfg: EXISTING configuration version 5 detected.

Clscfg: version 5 is 11g Release 2.

Successfully accumulated necessary OCR keys.

Clscfg: Arguments check out successfully.

No keys were written. Supply-force parameter to override.

-Force is destructive and will destroy any previous cluster

Configuration.

Failed to create voting files on disk group VOTE.

Change to configuration failed, but was successfully rolled back.

CRS-4000: Command Replace failed, or completed with errors.

Voting file add failed

Failed to add voting disks at/g01/11 ggrid/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 6930.

/G01/11 ggrid/app/11.2.0/grid/perl/bin/perl-I/g01/11 ggrid/app/11.2.0/grid/perl/lib-I/g01/11 ggrid/app/11.2.0/grid/crs/install/g01/11 ggrid/app/11.2.0/grid/crs/install/rootcrs. pl execution failed

This error indicates that the voting file cannot be created for the VOTE of the disk group. Failed to change configuration, but rolled back successfully.

Failed to add the voting file. failed to add the voting disk in/g01/11 ggrid/app/11.2.0/grid/crs/install.

The first error was encountered. It was very strange. Because there was no problem with mounting a disk using ASMLIB before, but this problem occurs during installation and cannot be continued.

Check ASM's alert. log to find:

ERROR: cocould not create voting files. It spans internal SS 161 AUs (max supported is 64 AUs)

ERROR: Voting file allocation failed for group VOTE

Errors in file/g01/11 ggrid/app/11.2.0/grid/diag/asm/+ ASM1/trace/+ ASM1_ora_7321.trc:

ORA-15303: Voting files cocould not be created in diskgroup VOTE due to small Allocation Unit size

View the corresponding DM of the arbitration disk of the corresponding GRID

# Multipath-ll

SC _vote (360a980003830302d712b31686276736b) dm-11 NETAPP, LUN

Size = 5.0G features = '4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle 'hwhandler = '0' wp = rw

'-+-Policy = 'round-robin 0' prio = 2 status = active

'-17: 0: 0: 0 sdax 67: 16 active ready running

Continue to check the size of the DM-11 block and find some information:

# Cd/sys/block/dm-11/queue

# More * block_size

::::::::::::::

Logical_block_size

::::::::::::::

512

::::::::::::::

Physical_block_size

::::::::::::::

4096

The reason is that it shows different logical and physical block sizes. In most cases, the size of physical and logical blocks is the same as "512 ".

How can this problem be solved? To query the official ORACLE website, the solution is as follows:

1. Oracle bug 11780656 is already fixed in 11.2.0.3 with compatible. asm = 11.2.0.3 but OUI does not allow to specify "compatible. asm" attribute to create OCRVOTE diskgroup.

-Bug 11780656-asm managed voting files cannot be more than 64X AU SIZE

2. Oracle bug 13999609 indicates that ASMLib driver only works with the expectation that logical block size and physical block size are 512/512 bytes.

-Bug 13999609-physical block size reported can cause issues with 10G DATABASES

SOLUTION

1] Possible workaround is to use '/dev/oracleasm/disks/*' path instead of "ORCL: *" when creating OCRVOTE diskgroup in OUI.

OR

2] Install the new oracleasm-support-2.1.8-1 asmlib rpm package (which contains the permanent fix) as described in note 1500460.1

As prompted above, we can either install the oracleasm-support-2.1.8-1 package or switch

/Dev/oracleasm/disks/* path to solve this problem. Solve the problem according to the preceding prompts and find that the GRID cannot be installed successfully.

After several times, no solution was found. Because the backend uses the NETAPP storage, go to the NETAPP official website to find the CASE. The following solutions are provided:

For details and caveats regarding this workaround, see the Oracle Alert.

Additionally, Oracle has provided a patch and configuration parameter to enable ASMlib to continue to function using the correct logical block size.

# ORACLEASM_USE_LOGICAL_BLOCK_SIZE: 'true' means use the logical block size
# reported by the underlying disk instead of the physical. The default
# is 'false'

(Oracle settings)

NetApp has also provided a workaround in versions 8.0.5, 8.1.3 and 8.2 of Data ONTAP 7-Mode. the workaround allows specified LUNs to continue to not report the logical blocks per physical block value. this work around und shoshould only be applied to LUNs used by Oracle ASMlib with the symptoms described in this article.

From the Data ONTAP 7-Mode CLI, enter the following commands:

> Lun set report-physical-size Disable (netapp storage settings)

Follow the prompts to go to the/etc/sysconfig directory and modify the oracleasm file under it.

[Oracle @ rac1 ~] $ Cat/etc/sysconfig/oracleasm

#

# This is a configuration file for automatic loading of the Oracle

# Automatic Storage Management library kernel driver. It is generated

# By running/etc/init. d/oracleasm configure. Please use that method

# To modify this file

#

# ORACLEASM_ENABLED: 'true' means to load the driver on boot.

ORACLEASM_ENABLED = true

# ORACLEASM_UID: Default user owning the/dev/oracleasm mount point.

ORACLEASM_UID = grid

# ORACLEASM_GID: Default group owning the/dev/oracleasm mount point.

ORACLEASM_GID = asmadmin

# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.

ORACLEASM_SCANBOOT = true

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning

ORACLEASM_SCANORDER = ""

# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan

ORACLEASM_SCANEXCLUDE = ""

# ORACLEASM_USE_LOGICAL_BLOCK_SIZE: 'true' means use the logical block size

# Reported by the underlying disk instead of the physical. The default

# Is 'false'

ORACLEASM_USE_LOGICAL_BLOCK_SIZE = false

Change ORACLEASM_USE_LOGICAL_BLOCK_SIZE = false to true.

Modify the storage settings on the NETAPP.

(Note: Oracle provides configuration parameters to enable ASMLib to continue using the correct logical block size function to avoid this problem .)

Through the above processing, the problem can be solved, and the GRID is finally installed successfully.