Environment Introduction: Dual-host operating system: solaris10 database version: oracle11gR164bit1. When I receive a call in the middle of the night, I say that the database reports a large number of errors. I can view the database and find that the database has crashed. view the alert Log and send
Environment Introduction: Dual-host operating system: solaris10 database version: oracle11gR164bit1. When I receive a call in the middle of the night, I say that the database reports a large number of errors. I can view the database and find that the database has crashed. view the alert Log and send
Environment Introduction:
Dual-host operating system: solaris 10 database version: oracle 11g R1 64bit
1. I received a call in the middle of the night, saying that the database reported a large number of errors. I checked the database and found the database crashed. I/O reported an error:
Wed Dec 18 00:36:57 2013KCF: write/open error block = 0x98abe online = 1 file = 89/dev/raw/raw03error = 27063 txt: 'svr4 Error: 5: i/O errorAdditional information:-1 Additional information: 8192 'wed Dec 18 00:36:57 2013KCF: write/open error block = 0x9d70f online = 1 file = 91/dev/raw/raw05error = 27063 txt: 'svr4 Error: 5: I/O errorAdditional information:-1 Additional information: 8192 'ic IC datafile offline due to write error onAutomatic datafile offline due to write error on
2. I had a problem with the current network because the engineering team encountered a line. I asked the IDC room staff if there is a project tonight. The IDC staff said that new devices will be connected to the SAN network tonight, when no one hits the line, check the local system log and report the following error:
Dec 17 23:33:10 fly-db01 scsi: [ID 107833 kern. warning] WARNING:/scsi_vhci/ssd @ fly6000c5d0008a0000006b131400440 (ssd28): Dec 17 23:33:10 fly-db01 SCSI transport failed: reason 'tran _ err': retrying commandDec 17 23:33:10 fly-db01 scsi: [ID 107833 kern. warning] WARNING:/scsi_vhci/ssd @ fly6000c5d0008a0000006b131400930 (ssd52 ):
3. Check the running status of other server databases, alert logs, operating system logs, and whether the attached file system is read-only ,, it is found that some databases have crashed, and the file system of some hosts has become read-only. The operating system log reports the lpfc error. This error has also occurred before. Generally, after an lpfc error occurs for a while, the file system will read-only
Lpfc error: fly008:/var/log # cat messages | grep lpfcDec 18 00:34:05 fly008 kernel: [10201542.768302] lpfc. 0: 0 :( 0): 0203 Devloss timeout on WWPN 21: 4g: 00: 0b: 5e: 6a: 18: 14 NPort x014400 Data: x40000 x1 x0Dec 18 00:34:07 fly008 kernel: [10201544.816750] lpfc. 0: 0 :( 0): 0203 Devloss timeout on WWPN 21: 4 h: 00: 0b: 5e: 6a: 18: 14 NPort x014500 Data: x0 x7 x0Dec 18 00:34:07 fly008 kernel: [102015 44.816802] lpfc. 0: 0 :( 0): 0203 Devloss timeout on WWPN 21: 4 k: 00: 0b: 5e: 6a: 18: 14 NPort x014600 Data: x0 x7 x0 file system read-only error: fly008 ~ # Df-hFilesystemSize Used Avail Use % Mounted on/dev/mapper/vg_fly008_app-lv_fly008_app99G 41G 53G 44%/home/fly008fly008 ~ # Cd/home/fly008fly008/home/fly008 # touch 1.txt touch: cannot touch '1.txt ': Read-only file system
4. Perform a rollback for the configuration tonight. The problem disappears. Restart the database. The database starts normally. When the application fails to be pulled, the following error is reported in the Application Log:
SQLErrorCode: 376 ORA-00376: file 92 cannot be read at this timeORA-01110: data file 92: '/dev/raw/raw06'
5. the alert Log in the database also reports related errors.
DDE: Problem Key 'ora 1110 'was flood controlled (0x5) (no incident) ORA-01110: Data File 92:'/dev/raw/raw06 '*** 05:04:16. 284ORA-12012: automatically execute job 226 error ORA-00372: The file 92ORA-06512 cannot be modified at this time: in "FLY. DELETE_FLY_EXCEPTION_INFO ", line 8ORA-06512: In line 1
6. Check the status of the data file and mark it as recover. You need to restore it.
SQL> SELECT file_name, file_id, tablespace_name, status, online_status FROM DBA_DATA_FILES ORDER BY TABLESPACE_NAME; FILE_NAMEFILE_ID TABLESPACE_NAME STATUSONLINE_STATUS/dev/raw/raw0692FLYAVAILABLERECOVER
7. Archiving is enabled for the database, and database backup is available to restore 92 files
# Su-oracle $ sqlplus/as sysdbaSQL> archive log list; SQL> recover datafile 92; SQL> alter database datafile 92 online
8. After recovery, the application is pulled up normally and the business test is normal.
This article is from the "yangyang Youqing" blog. Be sure to keep this source