Problem: 1. All clients cannot log on this morning, and the system prompts that the application server configuration is incorrect. Analysis and connection processing: 1. Check the client logs and discover database exceptions. Caused by: Java. SQL. sqlexception: Listener refused the connection with the following error: ORA-12528, TNS: Listener: all appropriate instances are blocking new connections The connection descriptor used by the client was: /// 192.168.2.245: 1521/eas54 At oracle. JDBC. Driver. databaseerror. throwsqlexception (databaseerror. Java: 111) At oracle. JDBC. Driver. databaseerror. throwsqlexception (databaseerror. Java: 260) At oracle. JDBC. Driver. t4cconnection. Logon (t4cconnection. Java: 386) At oracle. JDBC. Driver. physicalconnection. <init> (physicalconnection. Java: 438) At oracle. JDBC. Driver. t4cconnection. <init> (t4cconnection. Java: 164) At oracle. JDBC. Driver. t4cdriverextension. getconnection (t4cdriverextension. Java: 34) At oracle. JDBC. Driver. oracledriver. Connect (oracledriver. Java: 752) At java. SQL. drivermanager. getconnection (drivermanager. Java: 525) At java. SQL. drivermanager. getconnection (drivermanager. Java: 140) At com. Kingdee. Bos. SQL. Shell. kddriver. Connect (unknown source) ... 36 More According to the error message, the database cannot be connected, prompting that the instance cannot be connected. 2. Check the listening status, and the eas54 instance is in the blocking status. 3. Check the Alert Log and find an exception. Thread 1 advanced to log sequence 1397 Current log #1 seq #1397 mem #0: D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ redo01.log Mon Nov 30 18:44:23 2009 Process startup failed, error Stack: Mon Nov 30 18:44:23 2009 Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_psp0_3504.trc: ORA-27300 ORA-27301: OS Memory: overlapping I/O operations in progress. ORA-27302: Too Many too many? Skgpspawn Mon Nov 30 18:44:24 2009 Process j000 died, see its trace file Mon Nov 30 18:44:24 2009 Kkjcre1p: Unable to spawn jobq slave Process Mon Nov 30 18:44:24 2009 Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_cjq0_3648.trc: Mon Nov 30 19:00:53 2009 Process startup failed, error Stack: Mon Nov 30 19:00:53 2009 Errors in file D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump \ eas54_psp0_3504.trc: ORA-27300 ORA-27301: OS Memory: overlapping I/O operations in progress. ORA-27302: Too Many too many? Skgpspawn The database started at last night.ErrorI/O errors may occur due to garbled characters, but the OS and I/O characters are displayed. 4. Someone tried to restart the database this morning, Start Failed. Starting up Oracle RDBMS version: 10.2.0.1.0. System Parameters with non-default values: Processes = 200 Sessions = 225 S ga_max_size = 1367343104 _ Shared_pool_size = 318767104 Shared_pool_size = 318767104 _ Large_pool_size = 8388608 Large_pool_size = 8388608 _ Java_pool_size = 8388608 Java _ pool_size = 8388608 _ Streams_pool_size = 0 Streams_pool_size = 0 Spfile = D: \ oracle \ product \ 10.2.0 \ db_1 \ DBS \ spfileeas54.ora Nls_language = Simplified Chinese Nls_territory = China Sga_target = 1367343104 Control_files = D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control01.ctl, D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control02.ctl, D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control03.ctl DB _ block_size = 8192 _ Db_cache_size = 1023410176 DB _ cache_size = 838860800 Compatible = 10.2.0.1.0 Db_file_multiblock_read_count = 16 Db_recovery_file_dest = D: \ oracle \ product \ 10.2.0/flash_recovery_area Db_recovery_file_dest_size = 2147483648 Undo_management = auto Undo_tablespace = undotbs1 Remote_login_passwordfile = exclusive Db_domain = Dispatchers = (Protocol = TCP) (Service = eas54xdb) Job_queue_processes = 10 Audit_file_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ adump Background_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ bdump User_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ udump Core_dump_dest = D: \ oracle \ product \ 10.2.0 \ admin \ eas54 \ cdump Db_name = eas54 Open_cursors = 300 _ B _tree_bitmap_plans = false _ No_or_expansion = true Optimizer_index_cost_adj = 50 Optimizer_index_caching = 90 Pga_aggregate_target = 203423744 Mman started with PID = 4, OS id = 3952 Dbw0 started with PID = 5, OS id = 3968 Lgwr started with PID = 6, OS id = 3976 Ckpt started with PID = 7, OS id = 3992 SMON started with PID = 8, OS id = 4004 Reco started with PID = 9, OS id = 4020 Cjq0 started with PID = 10, OS id = 4044 Mmon started with PID = 11, OS id = 4056 Mmnl started with PID = 12, OS id = 4064 Tue Dec 01 09:21:28 2009 Starting up 1 dispatcher (s) for network address '(address = (partial = yes) (Protocol = TCP ))'... Starting up 1 shared server (s )... Pmon started with PID = 2, OS id = 3916 Tue Dec 01 09:21:29 2009 Alter database Mount exclusive ORA-214 signalled during: Alter database Mount exclusive... Psp0 started with PID = 3, OS id = 3948 5. sqlplus connects to the database instance and checks the instance status. C: \ Documents ents and Settings \ Administrator> set oracle_sid = eas54 C: \ Documents ents and Settings \ Administrator> sqlplus/As sysdba SQL * Plus: Release 10.2.0.1.0-production on Tuesday December 1 10:39:46 2009 Copyright (c) 1982,200 5, Oracle. All rights reserved. Connect: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0-Production With the partitioning, OLAP and Data Mining options SQL> select instance_name, status from V $ instance; Instance_name status -------------------------------------------------------- Eas54 started The instance is in the started status and has not been started successfully. 6. stop and start the database and find an error. SQL> startup Oracle instance started. Total system global area 1367343104 bytes Fixed size 1250884 bytes Variable Size 335546812 bytes Database buffers 1023410176 bytes Redo buffers 7135232 bytes ORA-00214: Control File 'd: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control02.ctl' Version 14018 inconsistent with file 'D: \ oracle \ product \ 10.2.0 \ oradata \ eas54 \ control01.ctl 'the version 14016 The error message indicates that the control file is inconsistent. It is estimated that the I/O fault last night caused the inconsistency of the control file. Control02.ctl version 14018> control01.ctl version 14016 7. Back up the three control files and overwrite control01.ctl and control03.ctl with control02.ctl. 8. Restart the database. The database can be started normally and the system returns to normal. 9. Check the logs in the event viewer of the operating system and find that the logs have been cleared. You cannot know whether I/O errors exist. Follow-up suggestions: 1. pay close attention to the I/O status of the server, check whether the disk is abnormal, and regularly check the operating system logs. 2. Make sure that the daily backup is normal and copied to another machine, just in case. Reproduced from: http://dev.kingdee.com/bbs/viewthread.php? Tid = 837 |