Abnormal connection number of weblogic92 connection pool

Some customers say they found that the current number of connections in the WEBLOGIC92 connection pool (active capacity) is less than the initial connection number (initial capacity) through connection pool monitoring. On the phenomenon, the intuition to customers is that connection pooling initialization is problematic and does not help them initialize as many connections as they need. But he also found that there were no problems with the other pool in several connection pool. Get the problem, I also suspect that this may be a bug in WebLogic, but then the log sent from the customer found that the problem connection was disable. After the investigation found that the problem is indeed related to this pool has been disable, then why is the pool after disable, this problem will occur?

First, let's take a look at why this pool was disable. Manually forced suspend connection pool, database shutdown, network instability and other factors are likely to become connection pool was disable inducement. From the customer's log, I can see a lot of the following exceptions,

1:java.net.socketexception: Pipe disconnected (errno:32)

2:weblogic.common.resourcepool.resourcedisabledexception:pool JDBC Data Source-0 is disabled, cannot allocate resources to applications.

According to the above exception, first with the customer to confirm the existence of a database shutdown, forced disable connection operations, these are rejected by the customer, the most likely reason is that the network is unstable, the network is good when bad, it is easy to cause weblogic connection pool to The connection to the database server was interrupted, causing the connection pool to be disable.

So why does a disconnect interrupt cause connection pool to be disable? Here are two parameters: Countoftestfailurestillflush, countofrefreshfailurestilldisable. These two parameters are in the WebLogic connection pool implementation because control whether, when flush or disable connection pool, two refers to the successive failed operations (test, refresh) after a few consecutive failures to flush or disable connection pool. Note: This is said to be continuous, not intermittent, after each successful operation (test, refresh), both values will be reset to 0. By default, both values are 2, that is, after a continuous failure of 3 (2+1) times, the connection pool is flush or disable. The difference is that flush is used to empty all connections in the connection pool (usually interrupted connection), while the pool state remains in the running state, and for the latter, the connection pool becomes suspend. For the client, the WebLogic will attempt to recreate the connection when it is connection,reserve from the pool, and if the connection is created successfully, the client can get the connection available. For a suspend state, the client Reserve connection request is rejected directly, and an exception is received as follows:

Weblogic.common.resourcepool.ResourceDisabledException:Pool JDBC Data Source-0 is disabled, cannot allocate resources to applications

A disable connection pool. Do we need to resume by hand? For example, the database for some reason, the sudden shutdown, after the database recovery, we need to manually to resume this pool? No need, WebLogic internal implementation of the connection pool of the self-health check function, for disable connection pool,weblogic every 5 seconds (default_scan_unit) to do a connection attempt (try to create a physical connection, If the connection is successful, then the connection is placed directly into the connection pool, and our problem is here, and we look at the specific reason through the following process of reproduction:

1: Configure the number of connections for a datasource,connection specific configuration as follows:

When 2:weblogic starts, we can see that the current capacity is 15, at which point connection pool has just been initialized and WebLogic will create a corresponding number of connections based on initial capacity. If we close the database and then get the connection through the test program, you will see that we are unable to get the connection (note that we have to select Testonreserve), repeat three times, and then monitor the connection pool again. Because after three test failures, the connection pool is disable (the state is suspend) as follows:

3: Restart database. Since the connection pool's self-test function is implemented within the WebLogic, for disabled connection pool,weblogic to make a connection attempt every 5 seconds, if the connection is successful, the new connection will be put into the connection pool, Resume the connection pool at the same time. By monitoring we can see that the connection pool status becomes running and the current capacity becomes 1,