[Case] database faults caused by Oracle Control File Corruption

Source: Internet
Author: User
[Case] database faults caused by Oracle Control File Corruption

Problem:
1. All clients cannot log on this morning, and the system prompts that the application server configuration is incorrect.

Analysis and connection processing:
1. Check the client logs and discover database exceptions.
Caused by: Java. SQL. sqlexception: Listener refused the connection with the following error:
ORA-12528, TNS: Listener: all appropriate instances are blocking new connections
The connection descriptor used by the client was:
/// 192.168.2.245: 1521/eas54

At oracle. JDBC. Driver. databaseerror. throwsqlexception (databaseerror. Java: 111)
At oracle. JDBC. Driver. databaseerror. throwsqlexception (databaseerror. Java: 260)
At oracle. JDBC. Driver. t4cconnection. Logon (t4cconnection. Java: 386)
At oracle. JDBC. Driver. physicalconnection. <init> (physicalconnection. Java: 438)
At oracle. JDBC. Driver. t4cconnection. <init> (t4cconnection. Java: 164)
At oracle. JDBC. Driver. t4cdriverextension. getconnection (t4cdriverextension. Java: 34)
At oracle. JDBC. Driver. oracledriver. Connect (oracledriver. Java: 752)
At java. SQL. drivermanager. getconnection (drivermanager. Java: 525)
At java. SQL. drivermanager. getconnection (drivermanager. Java: 140)
At com. Kingdee. Bos. SQL. Shell. kddriver. Connect (unknown source)
... 36 More

According to the error message, the database cannot be connected, prompting that the instance cannot be connected.

2. Check the listening status, and the eas54 instance is in the blocking status.

3. Check the Alert Log and find an exception.
Thread 1 advanced to log sequence 1397
Current log #1 seq #1397 mem #0: D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ redo01.log
Mon Nov 30 18:44:23 2009
Process startup failed, error Stack:
Mon Nov 30 18:44:23 2009
Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_psp0_3504.trc:
ORA-27300
ORA-27301: OS Memory: overlapping I/O operations in progress.
ORA-27302: Too Many too many? Skgpspawn

Mon Nov 30 18:44:24 2009
Process j000 died, see its trace file
Mon Nov 30 18:44:24 2009
Kkjcre1p: Unable to spawn jobq slave Process
Mon Nov 30 18:44:24 2009
Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_cjq0_3648.trc:

Mon Nov 30 19:00:53 2009
Process startup failed, error Stack:
Mon Nov 30 19:00:53 2009
Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_psp0_3504.trc:
ORA-27300
ORA-27301: OS Memory: overlapping I/O operations in progress.
ORA-27302: Too Many too many? Skgpspawn

The database started at last night.ErrorI/O errors may occur due to garbled characters, but the OS and I/O characters are displayed.

4. Someone tried to restart the database this morning, Start Failed.
Starting up Oracle RDBMS version: 10.2.0.1.0.
System Parameters with non-default values:
Processes = 200
Sessions = 225
S ga_max_size = 1367343104
_ Shared_pool_size = 318767104
Shared_pool_size = 318767104
_ Large_pool_size = 8388608
Large_pool_size = 8388608
_ Java_pool_size = 8388608
Java _ pool_size = 8388608
_ Streams_pool_size = 0
Streams_pool_size = 0
Spfile = D: \ oracle \ product \ 10.2.0 \ db_1 \ DBS \ spfileeas54.ora
Nls_language = Simplified Chinese
Nls_territory = China
Sga_target = 1367343104
Control_files = D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control01.ctl, D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control02.ctl, D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control03.ctl
DB _ block_size = 8192
_ Db_cache_size = 1023410176
DB _ cache_size = 838860800
Compatible = 10.2.0.1.0
Db_file_multiblock_read_count = 16
Db_recovery_file_dest = D: \ oracle \ product \ 10.2.0/flash_recovery_area
Db_recovery_file_dest_size = 2147483648
Undo_management = auto
Undo_tablespace = undotbs1
Remote_login_passwordfile = exclusive
Db_domain =
Dispatchers = (Protocol = TCP) (Service = eas54xdb)
Job_queue_processes = 10
Audit_file_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ adump
Background_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump
User_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ udump
Core_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ cdump
Db_name = eas54
Open_cursors = 300
_ B _tree_bitmap_plans = false
_ No_or_expansion = true
Optimizer_index_cost_adj = 50
Optimizer_index_caching = 90
Pga_aggregate_target = 203423744
Mman started with PID = 4, OS id = 3952
Dbw0 started with PID = 5, OS id = 3968
Lgwr started with PID = 6, OS id = 3976
Ckpt started with PID = 7, OS id = 3992
SMON started with PID = 8, OS id = 4004
Reco started with PID = 9, OS id = 4020
Cjq0 started with PID = 10, OS id = 4044
Mmon started with PID = 11, OS id = 4056
Mmnl started with PID = 12, OS id = 4064
Tue Dec 01 09:21:28 2009
Starting up 1 dispatcher (s) for network address '(address = (partial = yes) (Protocol = TCP ))'...
Starting up 1 shared server (s )...
Pmon started with PID = 2, OS id = 3916
Tue Dec 01 09:21:29 2009
Alter database Mount exclusive
ORA-214 signalled during: Alter database Mount exclusive...
Psp0 started with PID = 3, OS id = 3948

5. sqlplus connects to the database instance and checks the instance status.
C: \ Documents ents and Settings \ Administrator> set oracle_sid = eas54

C: \ Documents ents and Settings \ Administrator> sqlplus/As sysdba

SQL * Plus: Release 10.2.0.1.0-production on Tuesday December 1 10:39:46 2009

Copyright (c) 1982,200 5, Oracle. All rights reserved.

Connect:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0-Production
With the partitioning, OLAP and Data Mining options

SQL> select instance_name, status from V $ instance;

Instance_name status
--------------------------------------------------------
Eas54 started

The instance is in the started status and has not been started successfully.

6. stop and start the database and find an error.
SQL> startup
Oracle instance started.

Total system global area 1367343104 bytes
Fixed size 1250884 bytes
Variable Size 335546812 bytes
Database buffers 1023410176 bytes
Redo buffers 7135232 bytes
ORA-00214: Control File 'd: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control02.ctl'
Version 14018 inconsistent with file
'D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control01.ctl 'the version 14016
The error message indicates that the control file is inconsistent. It is estimated that the I/O fault last night caused the inconsistency of the control file.

Control02.ctl version 14018> control01.ctl version 14016

7. Back up the three control files and overwrite control01.ctl and control03.ctl with control02.ctl.

8. Restart the database. The database can be started normally and the system returns to normal.

9. Check the logs in the event viewer of the operating system and find that the logs have been cleared. You cannot know whether I/O errors exist.

Follow-up suggestions:
1. pay close attention to the I/O status of the server, check whether the disk is abnormal, and regularly check the operating system logs.
2. Make sure that the daily backup is normal and copied to another machine, just in case.

Reproduced from: http://dev.kingdee.com/bbs/viewthread.php? Tid = 837

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.