ORA-600 error resolution instance

1: a power outage occurred on the weekend. The newly configured Intranet Test Database active recovery uard environment is automatically disabled. The following error is reported after the master database is started this morning:

[Oracle @ db1 ~] $ Sqlplus/nolog

SQL * Plus: Release 11.2.0.1.0 Production on Mon Jun 17 17:00:57 2013

Copyright (c) 1982,200 9, Oracle. All rights reserved.

SQL> conn/as sysdba

Connected to an idle instance.

SQL> startup

ORACLE instance started.

Total System Global Area 1.3362E + 10 bytes

Fixed Size 2217952 bytes

Variable Size 6777997344 bytes

Database Buffers 6576668672 bytes

Redo Buffers 4960256 bytes

Database mounted.

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1],

[6], [301353], [301354], [], [], [], [], [], [], []

2: first, recover the database and then try to pull the database. The problem persists.

SQL> recover database;

Media recovery complete.

SQL> alter database open;

Alter database open

*

ERROR at line 1:

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1],

[6], [301353], [301354], [], [], [], [], [], [], []

3: view the alert Log and related trace files to determine whether problems may occur in the control file.

[Oracle @ db1 ~] $ Tail-f alert_db.log

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_ora_32462.trc:

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6], [301353], [301354], [], [], [], [], [], [], [], [], []

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_ora_32462.trc:

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6], [301353], [301354], [], [], [], [], [], [], [], [], []

ORA-600 signalled during: alter database open...

Trace dumping is refreshing Ming id = [cdmp_20130617170117]

Mon Jun 17 17:02:13 2013

Sweep [inc] [24153]: completed

Sweep [inc2] [24153]: completed

Mon Jun 17 17:02:38 2013

Alter database recover database

Media Recovery Start

Started logmerger process

Parallel Media Recovery started with 4 slaves

Mon Jun 17 17:02:38 2013

Recovery of Online Redo Log: Thread 1 Group 3 Seq 6 Reading mem 0

Mem #0:/u01/app/oracle/oradata/DB/onlinelog/o1_mf_3_8vpmjgoq _. log

Mem #1:/u01/app/oracle/flash_recovery_area/DB/onlinelog/o1_mf_3_8vpmkm9x _. log

Media Recovery Complete (db)

Completed: alter database recover database

Mon Jun 17 17:02:54 2013

Alter database open

Beginning crash recovery of 1 threads

Parallel recovery started with 3 processes

Started redo scan

Completed redo scan

Read 152 KB redo, 0 data blocks need recovery

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_ora_32462.trc (incident = 24154 ):

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6], [301353], [301354], [], [], [], [], [], [], [], [], []

Incident details in:/u01/app/oracle/diag/rdbms/db1/db/incident/incdir_24154/db_ora_32462_i24154.trc

Mon Jun 17 17:02:55 2013

Trace dumping is refreshing Ming id = [cdmp_20130617170255]

Aborting crash recovery due to error 600

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_ora_32462.trc:

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6], [301353], [301354], [], [], [], [], [], [], [], [], []

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_ora_32462.trc:

ORA-00600: internal error code, arguments: [kcratr_nab_less_than_odr], [1], [6], [301353], [301354], [], [], [], [], [], [], [], [], []

ORA-600 signalled during: alter database open...

Mon Jun 17 17:03:13 2013

Sweep [inc] [24154]: completed

Sweep [inc2] [24154]: completed

4: The trace control script is generated to restore the control file.

SQL> select open_mode from v $ database;

OPEN_MODE

--------------------

MOUNTED

SQL> alter database backup controlfile to trace as '/tmp/1. ctl ';

Database altered.

[Oracle @ db1 ~] $ Cat/tmp/1.ctl

Create controlfile reuse database "DB" NORESETLOGS FORCE LOGGING ARCHIVELOG

MAXLOGFILES 16

MAXLOGMEMBERS 3

MAXDATAFILES 100

MAXINSTANCES 8

# MAXLOGHISTORY 292

LOGFILE

GROUP 1 (

'/U01/app/oracle/oradata/DB/onlinelog/ow.mf_00008vpmdkd9_. log ',

'/U01/app/oracle/flash_recovery_area/DB/onlinelog/ow.mf_00008vpmdryt _. Log'

) SIZE 512 m blocksize 512,

GROUP 2 (

'/U01/app/oracle/oradata/DB/onlinelog/ow.mf_2_8vpmfqmw _. log ',

'/U01/app/oracle/flash_recovery_area/DB/onlinelog/o1_mf_2_8vpmhk2z _. Log'

) SIZE 512 m blocksize 512,

GROUP 3 (

'/U01/app/oracle/oradata/DB/onlinelog/ow.mf_3_8vpmjgoq _. log ',

'/U01/app/oracle/flash_recovery_area/DB/onlinelog/o1_mf_3_8vpmkm9x _. Log'

) SIZE 512 m blocksize 512

-- STANDBY LOGFILE

-- GROUP 4 (

-- '/U01/app/oracle/oradata/DB1/onlinelog/o1_mf_4_8vpq6nvy _. log ',

-- '/U01/app/oracle/flash_recovery_area/DB1/onlinelog/o1_mf_4_8vpq7wk8 _. Log'

--) SIZE 512 m blocksize 512,

-- GROUP 5 (

-- '/U01/app/oracle/oradata/DB1/onlinelog/ow.mf_5_8vpqbh6s _. log ',

-- '/U01/app/oracle/flash_recovery_area/DB1/onlinelog/ow.mf_5_8vpqcmbj _. Log'

--) SIZE 512 m blocksize 512,

-- GROUP 6 (

-- '/U01/app/oracle/oradata/DB1/onlinelog/o1_mf_6_8vp1_3rz _. log ',

-- '/U01/app/oracle/flash_recovery_area/DB1/onlinelog/o1_mf_6_8vp1_v5w _. Log'

--) SIZE 512 m blocksize 512,

-- GROUP 7 (

-- '/U01/app/oracle/oradata/DB1/onlinelog/ow.mf_7_8vpqgw0j _. log ',

-- '/U01/app/oracle/flash_recovery_area/DB1/onlinelog/o1_mf_7_8vpqhcql _. Log'

--) SIZE 512 m blocksize 512

DATAFILE

'/U01/app/oracle/oradata/DB/datafile/o1_mf_system_8vpm8hf3 _. dbf ',

'/U01/app/oracle/oradata/DB/datafile/o1_mf_sysaux_8vpm8hjq _. dbf ',

'/U01/app/oracle/oradata/DB/datafile/o1_mf_undotbs1_8vpm8hl4 _. dbf ',

'/U01/app/oracle/oradata/DB/datafile/o1_mf_users_8vpm8hn1 _. dbf'

Character set ZHS16GBK;

SQL> shutdown immediate

ORA-01109: database not open

Database dismounted.

ORACLE instance shut down.

SQL> startup nomount

ORACLE instance started.

Total System Global Area 1.3362E + 10 bytes

Fixed Size 2217952 bytes

Variable Size 6777997344 bytes

Database Buffers 6576668672 bytes

Redo Buffers 4960256 bytes

SQL> @/tmp/1.ctl;

Control file created.

5. After recreating the control file, recover the database and open the database successfully.

SQL> alter database open;

Alter database open

*

ERROR at line 1:

ORA-01113: file 1 needs media recovery

ORA-01110: data file 1:

'/U01/app/oracle/oradata/DB/datafile/o1_mf_system_8vpm8hf3 _. dbf'

SQL> recover database;

Media recovery complete.

SQL> alter database open;

Database altered.

SQL> select open_mode, database_role from v $ database;

OPEN_MODE DATABASE_ROLE

------------------------------------

READ WRITE PRIMARY

6: add temporary tablespace data files by observing alert logs

[Oracle @ db1 ~] $ Tail-f alert_db.log

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_m0020.381.trc:

ORA-25153: Temporary Tablespace is Empty.

Mon Jun 17 17:17:25 2013

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_j007_400.trc:

ORA-25153: Temporary Tablespace is Empty.

Errors in file/u01/app/oracle/diag/rdbms/db1/db/trace/db_j007_400.trc:

ORA-12012: error on auto execute of job 12696

ORA-25153: Temporary Tablespace is Empty.

ORA-06512: at "DBSNMP. BSLN_INTERNAL", line 2073

ORA-06512: at line 1

Mon Jun 17 17:18:20 2013

Alter tablespace temp add tempfile

Completed: alter tablespace temp add tempfile

SQL> alter tablespace temp add tempfile;

Tablespace altered.

7: Observe physical database and find the following problems

[Root @ db2 ~] # Su-oracle

[Oracle @ db2 ~] $ Sqlplus/nolog

SQL * Plus: Release 11.2.0.1.0 Production on Mon Jun 17 17:18:50 2013

Copyright (c) 1982,200 9, Oracle. All rights reserved.

SQL> conn/as sysdba

Connected to an idle instance.

SQL> startup

ORACLE instance started.

Total System Global Area 1.3362E + 10 bytes

Fixed Size 2217952 bytes

Variable Size 6777997344 bytes

Database Buffers 6576668672 bytes

Redo Buffers 4960256 bytes

Database mounted.

ORA-10458: standby database requires recovery

ORA-01196: file 1 is inconsistent due to a failed media recovery session

ORA-01110: data file 1:

'/U01/app/oracle/oradata/DB2/datafile/o1_mf_system_08oc7c38 _. dbf'

SQL> alter database open read only;

Alter database open read only

*

ERROR at line 1:

ORA-10458: standby database requires recovery

ORA-01196: file 1 is inconsistent due to a failed media recovery session

ORA-01110: data file 1:

'/U01/app/oracle/oradata/DB2/datafile/o1_mf_system_08oc7c38 _. dbf'

8: Close physical standby, start the hot backup of the system tablespace on the master database, copy the corresponding files to the specified location in the slave database, and re-Synchronize normally.

SQL> shutdown immediate

ORA-01109: database not open

Database dismounted.

ORACLE instance shut down.

SQL> alter tablespace system begin backup;

Tablespace altered.

[Oracle @ db1 datafile] $ pwd

/U01/app/oracle/oradata/DB/datafile

[Oracle @ db1 datafile] $ scp o1_mf_system_8vpm8hf3 _. dbf root @ db2:/tmp

SQL> alter tablespace system end backup;

Tablespace altered.