ORA-3136 Error Analysis -- warning inbound connection timed out

Summary

Warning: inbound connection timed out (ORA-3136)

I have never encountered this error before. This morning, one of the Oracle servers of the customer was very busy on MSN, and this error message frequently appeared in alert, resulting in connection failure. The following is an error message for the alert file:

......

Wed Feb 27 09:03:02 2008 Completed checkpoint up to RBA [0x184d. 2.10], SCN: 1203810646 Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:03:27 2008 Warning: inbound connection timed out (ORA-3136) Wed Feb 27 09:04:30 2008 Incremental checkpoint up to RBA [0x184d. e5a6.0], current log tail at RBA [0x184d. 43aaa. 0] Wed Feb 27 09:05:02 2008

......

This warning I have never touched before, but literally it can be seen that it should be connect time out, which was common in the past is ORA-12170. So I am wondering.

Preliminary analysis

1) CheckListener. oraAndSqlnet. oraParameter settings, no other exceptions found

$ Cat listener. ora ################ filename ......: listener. ora # name ..........: # date ..........: ############### admin_restrictions_listener = on listener = (address_list = (address = (Protocol = IPC) (Key = hs5.world )) (address = (Protocol = IPC) (Key = HS5) (address = (community = sap. world) (Protocol = TCP) (host = gvshs5db) (Port = 1527 ))) listener = 0 connect_timeout_listener = 10 trace_level_listener = off sid_list_listener = (sid_list = (sid_desc = (sid_name = HS5) (ORACLE_HOME =/Oracle/HS5/102_64) $ cat sqlnet. ora ################ filename ......: sqlnet. ora ################ automatic_ipc = on trace_level_client = off names. default_domain = world #05.01.06 unsorported parameter now # name. default_zone = world #05.01.06 set the default to 10 sqlnet. expire_time = 10 #05.01.06 set to default # TCP. nodelay = yes #05.01.06 set to 32768 default_sdu_size = 32768 $

2) through topas and vmstat, we can see that the current system has a high load, and the CPU is basically 100% (omitted)

Therefore, it can be preliminarily determined that the connection timeout is caused by excessive system load.

Further Analysis

I have not seen this warning before, so I directly consulted Oracle related materials. It turns out that this is a newly added attribute of 10gr2, which can be set through sqlnet. inbound_connect_timeout. The default value is 60 seconds.

The main cause of this warning problem may be:

1) server gets a connection request from a malicious (malicious) client which is not supposed to connect to the database, in which case the error thrown is the correct behavior. you can get the client address for which the error was thrown via sqlnet log file.
2) The server has es a valid client connection request but the client takes a long time to authenticate more than the default 60 seconds.
3) the DB server isHeavily LoadedDue to which it cannot finish the client logon within the timeout specified.

How can we locate the cause of this warning?

The default value of 60 seconds is good enough in most conditions for the database server to authenticate a client connection. if its taking longer period, then its worth checking all the below points before going for the workadound:

1. Check whether local connection on the database server is sucessful & quick.
2. If local connections are quick, then check for underlying network delay with the help of your network administrator.
3. Check whether your database performance has degraded by anyway.
4. Check Alert Log for any critical errors for EG, ORA-600 or ORA-7445 and get them resolved first.
These critical errors might have triggered the slowness of the database server.

As a workaround to avoid only this warning messages, you can set the parameters sqlnet. inbound_connect_timeout
And inbound_connect_timeout _ <listenername> to the value more than 60.

For e. g 120. So that the client will have more time to provide the authentication information to the database. You may have to further tune these parameter values according to your setup.

To set these parameter
1. In server side sqlnet. ora file add sqlnet. inbound_connect_timeout
For e. g
Sqlnet. inbound_connect_timeout = 120
2. In listener. ora file-inbound_connect_timeout _ <listenername >=110
For E. G if the listener name is listener then-
Inbound_connect_timeout_listener = 110

Note: from Oracle version 10.2.0.3 onwards the default value of inbound_connect_timeout _ <listenername> is 60 seconds. For previous releases it is zero by default.

How to check whether inbound timout is active for the listener and Database Server

For eg. inbound_connect_timeout _ <listener_name> = 4

You can check whether the parameter is active or not by simply doing telnet to the listener port.
$Telnet <Database Server IP> <listener port>

For eg.
$Telnet 192.168.12.13 1521
The telnet session shoshould disconnect after 4 seconds which indicates that the inbound connection timeout for the listener is active.

To check whether Database Server sqlnet. inbound_connect_timeout is active:
Eg.Sqlnet. inbound_connect_timeout = 5

A. For dedicated server setup, enable the support level sqlnet server tracing will show the timeout value as below:

Niotns: Enabling cto, value = 5000 (milliseconds) <= 5 seconds Niotns: Not enabling dead connection detection. Niotns: Listener bequeathed shadow coming to life...

B. For Shared Server Setup,
$ Telnet <Database Server IP> <dispatcher port>
For eg.
$Telnet 192.168.12.13 51658
The telnet session shoshould disconnect after 5 seconds which indicates that the sqlnet. inbound_connection_timeout is active.

from: http://tomszrp.itpub.net/post/11835/467438