Oracle 10g bare device data file recovery, ORA-19501, ORA-27063

Environment Description:
OS: aix 6.1
DB: bare Oracle 10gR2 Device

An error occurred while using rman to back up the entire database. If the 71 Data File fails to be backed up separately
$ Rman target/

Recovery Manager: Release 10.2.0.1.0-Production on Thu Nov 7 09:14:31 2013

Copyright (c) 1982,200 5, Oracle. All rights reserved.

Connected to target database: ORCL (DBID = 1237792397, not open)

RMAN> backup as copy datafile 71 format'/backup_dir/in71 _ % U ';

Starting backup at 07-NOV-13
Using target database control file instead of recovery catalog
Allocated channel: ORA_DISK_1
Channel ORA_DISK_1: sid = 1086 devtype = DISK
Channel ORA_DISK_1: starting datafile copy
Input datafile fno = 00071 name =/dev/rdata_03
RMAN-00571: ========================================================== ==============================
RMAN-00569: ==================== error message stack follows ==========================
RMAN-00571: ========================================================== ==============================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 11/07/2013 09:18:10
ORA-19501: read error on file "/dev/rdata_03", blockno 178305 (blocksize = 8192)
ORA-27063: number of bytes read/written is incorrect
Ibm aix risc System/6000 Error: 5: I/O error
Additional information:-1
Additional information: 1048576

RMAN> backup check logical validate datafile 71;

Starting backup at 07-NOV-13
Using channel ORA_DISK_1
Channel ORA_DISK_1: starting full datafile backupset
Channel ORA_DISK_1: specifying datafile (s) in backupset
Input datafile fno = 00071 name =/dev/rdata_03
RMAN-00571: ========================================================== ==============================
RMAN-00569: ==================== error message stack follows ==========================
RMAN-00571: ========================================================== ==============================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 11/07/2013 09:47:03
ORA-19501: read error on file "/dev/rdata_03", blockno 178305 (blocksize = 8192)
ORA-27063: number of bytes read/written is incorrect
Ibm aix risc System/6000 Error: 5: I/O error
Additional information:-1
Additional information: 1048576

Query the number of Bad blocks
SQL> select * from v $ database_block_corruption;

No rows selected

No logical Bad blocks are found. I/O error is returned when I think the database is a good database and the Physical Bad blocks are guessed.

Check the number of data blocks in File 71
SQL> select bytes/8192 from v $ datafile where file # = 71;

BYTES/8192
----------
1280000


Verify whether the bare device can read
$ Dbv file =/dev/rdata_03 blocksize = 8192

DBVERIFY: Release 10.2.0.1.0-Production on Thu Nov 7 09:48:08 2013

Copyright (c) 1982,200 5, Oracle. All rights reserved.

DBVERIFY-Verification starting: FILE =/dev/rdata_03

DBV-00102: File I/O error on FILE (/dev/rdata_03) during verification read operation (-2)

 

Find out the bad blocks and use the tool provided by Oracle
$ Dbv file =/dev/rdata_03 blocksize = 8192

DBVERIFY: Release 10.2.0.1.0-Production on Thu Nov 7 09:48:08 2013

Copyright (c) 1982,200 5, Oracle. All rights reserved.

DBVERIFY-Verification starting: FILE =/dev/rdata_03

DBV-00102: File I/O error on FILE (/dev/rdata_03) during verification read operation (-2)
View
$ Ps-ef | grep dbv
Oracle 598170 593980 38 09:57:42 pts/1 dbv file =/dev/rdata_03 blocksize = 8192
Oracle 1712282 581750 0 09:57:46 pts/0 grep dbv
$ Truss-faieo/tmp/strace. out-p 598170

598170: klseek (5, 746045440, 0, 0x0FFFFFFFFFFF9530) = 0
... Skipping...
598170: kread (5, "(0 \ 011 ??? D 5 ??.., 8192) = 8192
598170: klseek (5, 1461190656, 0, 0x0FFFFFFFFFFF9530) = 0
598170: kread (5, "(0 \ 011 ??? D 5 ??.., (8192) Err #5 EIO
598170: kwrite (2, "\ n d B V-0 0 1 0 2:", 12) = 12
598170: kwrite (2, "F I l e I/O e r"..., 24) = 24
598170: kwrite (2, "/d e v/r d a t a _ c"..., 15) = 15
598170: kwrite (2, ") d u r I n g", 9) = 9
598170: kwrite (2, "v e r I f I c a t I o n"..., 17) = 17
598170: kwrite (2, "o p e r a t I o n (", 12) = 12
598170: kwrite (2, "-", 1) = 1
598170: kwrite (2, "2", 1) = 1
598170: kwrite (2, ")", 1) = 1
598170: kwrite (2, "\ n", 1) = 1
598170: close (5) = 0
598170: close (4) = 0
598170: kfcntl (1, F_GETFL, 0x00000000101A2087) = 67110914
598170: kfcntl (2, F_GETFL, 0x00000000101A2087) = 67110914
598170: _ exit (1)
... Skipping...
598170: kread (5, "(0 \ 011 ??? D 5 ??.., 8192) = 8192
598170: klseek (5, 1461190656, 0, 0x0FFFFFFFFFFF9530) = 0
598170: kread (5, "(0 \ 011 ??? D 5 ??.., (8192) Err #5 EIO
598170: kwrite (2, "\ n d B V-0 0 1 0 2:", 12) = 12
598170: kwrite (2, "F I l e I/O e r"..., 24) = 24
598170: kwrite (2, "/d e v/r d a t a _ c"..., 15) = 15
598170: kwrite (2, ") d u r I n g", 9) = 9
598170: kwrite (2, "v e r I f I c a t I o n"..., 17) = 17
598170: kwrite (2, "o p e r a t I o n (", 12) = 12
598170: kwrite (2, "-", 1) = 1
598170: kwrite (2, "2", 1) = 1
598170: kwrite (2, ")", 1) = 1
598170: kwrite (2, "\ n", 1) = 1
598170: close (5) = 0
598170: close (4) = 0
598170: kfcntl (1, F_GETFL, 0x00000000101A2087) = 67110914
598170: kfcntl (2, F_GETFL, 0x00000000101A2087) = 67110914
598170: _ exit (1)
Find the starting Bad Block
1461190656/8192 = 178368
It indicates that the first bad block is 178368, And the last bad block is verified. The maximum data block is 1280000.

$ Dbv file =/dev/rdata_03 blocksize = 8192 start = 1461190656

DBVERIFY: Release 10.2.0.1.0-Production on Thu Nov 7 10:10:58 2013

Copyright (c) 1982,200 5, Oracle. All rights reserved.

DBV-00102: File I/O error on FILE (/dev/rdata_03) during start read operation (-2)

$ Dbv file =/dev/rdata_03 blocksize = 8192 start = 194719

DBVERIFY: Release 10.2.0.1.0-Production on Thu Nov 7 10:52:56 2013

Copyright (c) 1982,200 5, Oracle. All rights reserved.

DBVERIFY-Verification starting: FILE =/dev/rdata_03

DBVERIFY-Verification complete

Total Pages Examined: 1085282
Total Pages Processed (Data): 1272
Total Pages Failing (Data): 0
Total Pages Processed (Index): 127
Total Pages Failing (Index): 0
Total Pages Processed (Other): 1082867
Total Pages Processed (Seg): 0
Total Pages Failing (Seg): 0
Total Pages Empty: 1016
Total Pages Marked success upt: 0
Total Pages Influx: 0
Highest block SCN: 1845997064 (2919.1845997064)

Reconstructs the data file in the Data splicing mode.

$ Dd if =/dev/zero of =/backup_dir/rdata_03_02 bs = 8192 count = 16343
16343 + 0 records in.
16343 + 0 records out.
$ Dd if =/dev/rdata_03 of =/backup_dir/rdata_03_03 bs = 8192 skip = 194710
1116010 + 0 records in.
1116010 + 0 records out.
$ Dd if =/dev/rdata_03 of =/backup_dir/rdata_03_01 bs = 8192 count = 178367
178367 + 0 records in.
178367 + 0 records out.
$ Cat/backup_dir/rdata_03_01/backup_dir/rdata_03_02/backup_dir/rdata_03_03>/backup_dir/rdata_03_ct

For more details, please continue to read the highlights on the next page:

SPFILE error causing database startup failure (ORA-01565)

ORA-01172, ORA-01151 error handling

ORA-00600 [2662] troubleshooting

Troubleshooting for ORA-01078 and LRM-00109

Notes on ORA-00471 Processing Methods

ORA-00314, redolog corruption, or missing Handling Methods

Solution to ORA-00257 archive logs being too large to store

1 2 3 Next Page