ORA-00245 troubleshooting

Today, we can see the following problems in the daily response of data centers A and B:

Renewal point 1 day:

Tue Jul 31 03:02:50 2012

Errors in file/soft/Oracle/diag/rdbms/a1qzdb/a1qzdb1/trace/a1qzdb1_ora_5374100.trc:

ORA-00245: control file backup operation failed

Tue Jul 31 09:08:37 2012

Thread 1 advanced to log sequence 346 (LGWRswitch)

Current log #1 seq #346 mem #0: + DATA/a1qzdb/onlinelog/group_1.259.786961989

Tue Jul 31 09:08:37 2012

LNS: Standby redo logfile selected forthread 1 sequence 346 for destination LOG_ARCHIVE_DEST_2

Tue Jul 31 09:08:40 2012

Archived Log entry 1132 added for thread 1 sequence 345 ID 0x1f7e84e dest 1:

Tue Jul 31 09:27:34 2012

Control file backup creation failed.

Backup target file size found to be zero.

Errors in file/soft/oracle/diag/rdbms/a1qzdb/a1qzdb1/trace/a1qzdb1_arc2_15073292.trc:

ORA-27037: unable to obtain file status

Ibm aix risc System/6000 Error: 2: No such file or directory

Additional information: 3

Renewal Point 2:

Tue Jul 31 03:02:50 2012

Control file backup creation failed.

Backup target file size found to be zero.

Errors in file/soft/oracle/diag/rdbms/a1qzdb/a1qzdb2/trace/a1qzdb2_ckpt_760261_trc:

ORA-27037: unable to obtain file status

Ibm aix risc System/6000 Error: 2: No suchfile or directory

Additional information: 3

Tue Jul 31 09:16:27 2012

Thread 2 advanced to log sequence 284 (LGWRswitch)

Current log #6 seq #284 mem #0: + DATA/a1qzdb/onlinelog/group_6.275.786964219

Tue Jul 31 09:16:27 2012

LNS: Standby redo logfile selected forthread 2 sequence 284 for destination LOG_ARCHIVE_DEST_2

Tue Jul 31 09:16:30 2012

Archived Log entry 1134 added for thread 2 sequence 283 ID 0x1f7e84e dest 1:

Tue Jul 31 09:27:29 2012

Expanded controlfile section 19 from 818 to1636 records

Requested to grow by 818 records; added 2 blocks of records

Tue Jul 31 09:27:34 2012

Errors in file/soft/oracle/diag/rdbms/a1qzdb/a1qzdb2/trace/a1qzdb2_ora_15204484.trc:

ORA-00245: control file backup operation failed

Corresponding trace daily values:

/Soft/oracle/diag/rdbms/a1qzdb/a1qzdb1/trace/a1qzdb1_ora_5374100.trc:

* ** 03:02:50. 674

* ** Session id: (541.7217) 2012-07-3103: 02: 50.675

* ** Client id: () 03:02:50. 675

* ** Service name :( SYS $ USERS) 2012-07-3103: 02: 50.675

* ** Module name :( rman @ qza1db1 (TNS V1-V3) 03:02:50. 675

* ** Action name: (0000011 STARTED114) 03:02:50. 675

 

/Soft/oracle/diag/rdbms/a1qzdb/a1qzdb1/trace/a1qzdb1_arc2_15073292.trc:

ORA-00245: control file backup operationfailed

 

* ** 09:27:34. 107

Ibm aix risc System/6000 Error: 2: No suchfile or directory

Additional information: 3

 

/Soft/oracle/diag/rdbms/a1qzdb/a1qzdb2/trace/a1qzdb2_ckpt_760261_trc:

* ** 09:27:34. 382

* ** Session id: (885.58917) 2012-07-3109: 27: 34.383

* ** Client id: () 09:27:34. 383

* ** Service name :( SYS $ USERS) 2012-07-3109: 27: 34.383

* ** Module name :( rman @ qza1db2 (TNS V1-V3) 09:27:34. 383

* ** Action name: (0000021 STARTED114) 09:27:34. 383

 

ORA-00245: control file backup operationfailed

 

/Soft/oracle/diag/rdbms/a1qzdb/a1qzdb2/trace/a1qzdb2_ora_15204484.trc:

* ** 03:02:50. 508

Ibm aix risc System/6000 Error: 2: No suchfile or directory

Additional information: 3

 

After a Google attack, we found that this is a "feature" after Oracle11.2.0.2.0 ":

Starting from oracle 11.2.0.2.0,The Snapshot file of the control file must be accessible to data instances of all nodes and placed in a shared location.. Otherwise, the following error is reported when you use RMAN to back up the control file:

ORA-00245: control file backup operationfailed.

When backup scripts are used for backup, this error may cause backup interruption.

Two documents in METALINK:

Rac backup fails with ORA-00245: controlfile backup operation failed [ID1268725.1] [not found]

RMAN Snapshot Controlfile Must Reside onShared Device for RAC database in 11G [ID 1263621.1] [not found]

Explained and solved this problem! Abstract:

"RMAN creates a copy of the controlfile for read consistency, this is the snapshot controlfile. due to the changesmade to the controlfile backup mechanism in 11gR2 any instances in the clustermay write to the snapshot controlfile. therefore, the snapshot controlfile fileneeds to be visible to all instances.

The same happens when a backup of thecontrolfile is created directly from sqlplus any instance in the cluster maywrite to the backup controfile file.

In 11gR2 onwards, the controlfile backuphappens without holding the control file enqueue. For non-RAC database, thisdoesn't change anything.

But, for RAC database, the snapshotcontrolfile location must be in a shared file system that will be accessible fromall the nodes ."

 

Current Control File Configuration:

RMAN> show all;

 

Using target database control file insteadof recovery catalog

RMAN configuration parameters for databasewith db_unique_name OSDB are:

Configure retention policy to redundancy 1; # default

Configure backup optimization off; # default

Configure default device type to disk; # default

Configure controlfile autobackup off; # default

Configure controlfile autobackup format fordevice type disk to '% F'; # default

Configure device type disk parallelism 1 backup type to backupset; # default

Configure datafile backup copies for devicetype disk to 1; # default

Configure archivelog backup copies fordevice type disk to 1; # default

Configure maxsetsize to unlimited; # default

Configure encryption for database off; # default

Configure encryption algorithm 'aes128 '; # default

Configure compression algorithm 'basic 'asof release 'default' optimize for load true; # DEFAULT

Configure archivelog deletion policy tonone; # default

Configure snapshot controlfile name to '/soft/product/11.2.0.2/dbh/dbs/snapcf_osdb1.f'; # default

 

Since we are using ASM, we can use the following configuration to solve this problem, or we can convert ASM into a shared object:

RMAN>Configure snapshot controlfile name to '+ SFRA/sncforcl_osdb.ora ';

 

New RMAN configuration parameters:

Configure snapshot controlfile name to '+ SFRA/sncforcl_osdb.ora ';

New RMAN configuration parameters aresuccessfully stored

 

RMAN> show all;

 

RMAN configuration parameters for databasewith db_unique_name OSDB are:

Configure retention policy to redundancy 1; # default

Configure backup optimization off; # default

Configure default device type to disk; # default

Configure controlfile autobackup off; # default

Configure controlfile autobackup format fordevice type disk to '% F'; # default

Configure device type disk parallelism 1 backup type to backupset; # default

Configure datafile backup copies for devicetype disk to 1; # default

Configure archivelog backup copies fordevice type disk to 1; # default

Configure maxsetsize to unlimited; # default

Configure encryption for database off; # default

Configure encryption algorithm 'aes128 '; # default

Configure compression algorithm 'basic 'asof release 'default' optimize for load true; # DEFAULT

Configure archivelog deletion policy tonone; # default

Configure snapshot controlfile name to '+ SFRA/sncforcl_osdb.ora ';

 

Attachment:

In RAC environment from 11.2 onwards Backup Or Snapshot controlfile needs to be in shared location [ID 1472171.1]
	Modification time13-JUL-2012TypeALERTStatusPUBLISHED

In this Document

Description

 

Occurrence

 

Symptoms

 

Workaround

 

Patches

 

History

 

References

Applies:

Oracle Server-Enterprise Edition-Version 11.2.0.1.0 and later
Information in thisdocument applies to any platform.
This issue is onlyapplicable to RAC database.

From 11gR2 onwards, the controlfile backup happens without holding the controlfile enqueue. fornon-RAC database, this doesn' t change anything. but for RAC database, due tothe changes made to the controlfile backup mechanism in 11gR2, any instance inthe cluster may write to the snapshot controlfile. due to this snapshotcontrolfile need to be visible to all instances.

The snapshotcontrolfile MUST be accessible by all nodes of a RAC database, if the snapshotcontrolfile does not reside on a shared device error will be raised at the timeof RMAN backup while taking snapshot of controlfile.

This applies tobacking up controlfile using sqlplus/having autobackup of controlfileconfigured on non
Shared location.

ORA-245 errormessage description
----------------------------------------
00245,000 00, "control file backup operation failed"
* Cause: Failed tocreate a control file backup because some process
Signaled an errorduring backup creation.
* Action: Checkalert files for further information. This usually happens
Because someprocess cocould not access the backup file
Backup creation. Any process of any instance that starts
Read/write controlfile transaction must have an access to
Backup control fileduring backup creation.

Description

1. In RACenvironment controlfile autobackup fails with ora-0245

Autobackupof controlfile in RMAN is failing with error:
RMAN-571: ========================================================== ==============================
RMAN-569: ==================== error message stack follows ==========================
RMAN-571: ========================================================== ==============================
RMAN-3009: failure of Control File and SPFILE Autobackup command on
ORA_DISK_1 channel at 10/27/2010 12:13:31
ORA-245: control file backup operation failed

2. In RAC environment, backupcontrolfile to non shared location fails

SQL> ALTER DATABASE BACKUP CONTROLFILE TO '/home/rac1122/test/control. bk' REUSE
*
ERROR at line 1:
ORA-245: control file backup operation failed

3. In RAC environment backing upstandby controlfile to non shared location fails

SQL> alter database create standby controlfile as '/home/oracle/renostdbycntrl. ctl ';

Alterdatabase create standby controlfile
'/Home/oracle/renostdbycntrl. ctl'
*
ERROR at line 1:
ORA-245: control file backup operation failed

4. In RAC environment copy currentcontrolfile to '$ {DB_BACKUP_DIR}/rac_tnctv_control.bak ';

Channelmetadata: starting datafile copy
Copying current control file
RMAN-571: ========================================================== ==============================
RMAN-569: ==================== error message stack follows ==========================
RMAN-571: ========================================================== ==============================
RMAN-3009: failure of backup command on 10/07/2011 11:36:42 channel at least
ORA-245: control file backup operation failed

5. In RAC environment, Rman backupfails if snapshot controlfile is not in shared location.

RMAN-00571: ========================================================== ======================
RMAN-00569: ============== error message stack follows ==================
RMAN-00571: ========================================================== ========================
RMAN-03009: failure of resync command on default channel at 03/13/2012 10:19:41
ORA-00245: control file backup operation failed

Occurrence

Only affect Real application Cluster (RAC), 11.2 specific.

Symptoms

In RAC environment any form of controlfile backup may be failwith ORA-0245 if the location of the Snapshot Controlfile is not a sharedlocation.

The backup of the controlfile actualy makes a backup of theSNAPSHOT controlfile. The Snapshot controlfile is created when the controlfileis about to be backed up.

The Snapshot controlfile is a read-consistent copy of thecontrolfile.

 

Workaround

It is changed behaviour which requires that the snapshotcontrolfile in a RAC environment, is on a shared location.

1. Check the snapshot controlfile location:

RMAN> show snapshot controlfile name;

2. Configure the snapshot controlfile to a shared disk:

RMAN>CONFIGURE SNAPSHOT CONTROLFILE NAME TO'<shared_disk>/snapcf_<DBNAME>.f';

Or in case of ASM use

RMAN>CONFIGURE SNAPSHOT CONTROLFILE NAME TO'+<DiskGroup>/snapcf_<DBNAME>.f';

Patches

NA

History

02-07-2012created and Published alert