Fial over of DG master database, forced activation of slave database solution case

In March January 2014, the database needs to be moved from the original storage platform to the new storage platform due to changes in the hardware environment. That is, change the underlying storage medium of the database. The following describes the accident occurrence and countermeasures.

Overview

Win platform, 11R2, 64-bit, single instance, physical standby database of DG. Both the master and slave databases only have the fusion iocard as the storage medium for redo and business data files, and other data files and control files are stored on non-fusion iocard media. You need to replace the storage media fusion io card with the virident card. The two cards are directly inserted in the pci slot, and the two cards are produced by the same manufacturer. Most importantly, some people directly use the file copy method for database migration during previous tests. In view of the fast copy speed of the two types of cards, the direct data file card is used for card copy operations, and then the drive letter is changed to the same as the original, the card copy speed is very fast, I forgot to copy the data files stored on the fusion io card to the virident card every several hundred MB per second. Other data files and control files do not need to be moved.

Incident

As a result, all services are switched off and the master and slave databases are shut down. Start to copy files to the card for the master database. Soon the copy is complete, and the drive letter is changed. When the master database is started, the following error occurs:
12345678 Errors in file E: \ APP \ ADMINISTRATOR \ diag \ rdbms \ branch_p \ branch \ trace \ branch_lgwr_2196.trc:
ORA-00313 :??????? 3 (???? 1 )???
ORA-00312 :???? 3 ?? 1: 'J: \ REDO \ BRANCH \ REDO03.LOG'
ORA-01382 :?? 1 ???????? J: \ REDO \ BRANCH \ REDO03.LOG ????????? (4096 )???????? (512)
Errors in file E: \ APP \ ADMINISTRATOR \ diag \ rdbms \ branch_p \ branch \ trace \ branch_lgwr_2196.trc:
ORA-00313 :??????? 3 (???? 1 )???
ORA-00312 :???? 3 ?? 1: 'J: \ REDO \ BRANCH \ REDO03.LOG'
ORA-01382 :?? 1 ???????? J: \ REDO \ BRANCH \ REDO03.LOG ????????? (4096 )???????? (512)

Despite garbled characters, the error code shows that the log group cannot be opened. It is found that the log group cannot be opened as current. The ora-01382 shows that the block size of the log file is larger than the disk sector size of the virident card. Therefore, the log group cannot be opened.
At first, I didn't notice the ora-01382 error. I thought I 'd try to pull the Library first, so I used the command alter database clear unarchived logfile group 3;

However, the above error is still reported. Later found to solve the ORA-01382 this error, with alter system set "_ disk_sector_size_override" = true; change the parameter to solve the startup error cannot open the log problem, and then continue to start the database, the results can start normally, however, the database still executes the previous clear log operation. This clear operation will cause data loss, so cancel will be executed quickly. Then, clear all the data copied to the virident card and re-copy the data on the fusion iocard. Then, set the implicit parameter _ disk_sector_size_override after the database startup mount, and then open the database.

The data on the virident card has been cleared, and the original data file has been copied again. As we thought during the copy process, the control file's SCN has changed before the clear log. The data file header scn on the fusion iocard is inconsistent with the control file scn at this time, and the database cannot be opened. The log switches (that is, the generated archivelog) that occurred during the previous clear log are also cleared, so that the archived log files are not continuous. There is a gap between the logs of the slave database and the master database.

Troubleshooting

There are two solutions. An rman backup is made before the file is copied. This backup is a backup made before the card replacement is prepared when the business is stopped. Therefore, data with this backup will never be lost, we can use the rman backup file for restoration. Another method is to ensure that all business data is transferred to the slave database before the card swap, so you can pull the slave database as the master database. At this time, the DG Environment is fail over of the master database, and everything in the slave database is intact.

Determine the solution for the next step.

The recovery speed of rman backup files is too slow. We are less than three hours away from normal business. Then we decided to pull the slave database up and use this method as the master database. However, the Standby database also needs to move the data on the fusion iocard to the virident card, or copy files. Save time.

This time, all data files are statically backed up before the copy (the database has stopped) to prevent the primary database from re-occurring errors. After the backup, physically copy the data on the fusion iocard to the virident card and swap the drive letters. Then, the database startup mount, alter system set "_ disk_sector_size_override" = true; then open read only. It can be opened normally. Okay. The storage media is replaced successfully. The next step is to activate the slave database as the master database. It is to force activate the slave database, start the slave database to mount, and run the alter database activate standby database command to open the slave database, restart the slave database again. Connect to the Business System for normal use. Then shut down and unplug fusion io. By the normal business operation time, this database can be used as a production database. The original master database is discarded and a new physical standby is created.

Conclusion: No matter what backup is done first. Back up the data before you proceed. Exercise and handle problems calmly before doing things.

References:

Important configuration parameters of Oracle Data Guard

Configure Oracle 11g Data Guard based on the same host

Explore Oracle 11g elastic uard

Oracle Data Guard (RAC + DG) archive deletion policies and scripts

Role conversion for Oracle Data Guard

FAL gap in Oracle Data Guard logs

Oracle 11g Data Guard Error 16143 Heartbeat failed to connect to standby