Apache common-pool, common-DBCP source code interpretation and Object pool Principle Analysis

Http://teamojiao.iteye.com/blog/456851

I recently came into contact with the connection pool and Object pool technology in the optimization of an internal test tool class. I changed the original database access operations not using the connection pool to the connection pool method. the performance has been greatly improved. It turns out that after two transformations, it would take more than 500 seconds for a large test class. After the first optimization, it would only take more than 300 seconds, after switching to the connection pool for the second time, it takes more than 80 seconds for the same test class. the following is a summary of the transformation process.
The object pool is a common cache mechanism that uses "space for Time". The "time" here refers to the creation time. Therefore, this also gives the applicability of the Object pool: if the creation process of an object is time-consuming, use the object pool. simply put, the internal principle is to put the created object into a container, and then put it back into the container instead of destroying it, so that other objects can be called, the object pool also involves some advanced technologies, such as destruction upon expiration, destruction upon destruction, destruction when the number of objects exceeds the size of the pool, and waiting when no idle objects are available in the object pool.

Apache's common-pool tool library is a concrete implementation of the pooling technical principle. Before explaining it, I should first understand several concepts:
Object pool(Objectpool Interface): it can be considered as a container. It is used to hold pool objects and contains factory objects used to create pool objects.
Pool object: Objects to be placed in the pool container. In theory, they can be any object.
Object pool Factory(Objectpoolfactory Interface): used to create the factory of the Object pool.
Pool object Factory(Poolableobjectfactory Interface): This interface is used to create a pool object, deactivate unused pool objects (passivateobject), and activate (activeobject) the pool object to be used ), validate the pool object and destroy the problematic pool object.

The object pool encapsulates the responsibility for creating, obtaining, returning, and destroying pool objects. Of course, these tasks are implemented through the pool object factory, the container contains one or more containers used to hold the pool objects. the object pool controls the container size, storage time, access wait time, and idle time, because these settings can be adjusted as needed.

When you need to get a pool object, retrieve one from the container. If the container does not exist and does not meet the maximum container restrictions, the pool object factory is called, create a new pool object and call the factory Activation Method to activate and verify the created object. if the maximum value of the Pool container is reached, and no idle objects exist in the object pool, the system continues to wait until new idle objects are lost, of course, this wait is also limited. If the limit is exceeded, the object pool will throw an exception.

"Come out and mix, always pay back", the same is true for pool objects. When the used pool objects are returned to the object pool, the object pool calls the pool object factory to verify the pool object. If the verification fails, it is considered a problematic object and will be destroyed. If the container is full, this return pool object will become "Homeless" and will be destroyed. If it does not belong to the above two situations, the object pool will call the factory object to deactivate it and put it into the container. in the whole process, activation, check, and passivation are not necessary. Therefore, when we implement the poolableobjectfactory interface, we generally do not process it. We just need to leave it empty, so basepoolableobjectfactory is born.

Of course, you can also create existing objects and put them in the object pool through addobject for later use.

To ensure thread-safe access to the object pool, all operations on the container must be placed in synchronized.

There are five object pools in the Apache common-pool tool Library: genericobjectpool and generickeyedobjectpool, softreferenceobjectpool, stackobjectpool, and stackkeyedobjectpool.
There are two types of object pools:

Another type is key:

 

The first two methods use cursorablelinkedlist as the container, softreferenceobjectpool uses arraylist as the container, creates all pooled objects at a time, and processes softreference on objects in the container, this ensures that when the memory is sufficient, the pool object will not be easily reclaimed by JVM garbage, so as to have a strong cache capability. the last two types Use stack as containers. an object pool without keys is a simple implementation of the previous pool technical principle. The object pool with keys is more complex. It classifies the pool objects by keys, the same key is divided into a group of classes, so how many keys are there and how many containers are there. this type of object pool with key is required because the objects created by the makeobject () method in the general object pool are basically the same, because the parameters cannot be passed to customize the pool objects. therefore, the differences between the four pool objects are mainly reflected in the differences between internal containers. Stack follows the "first-in-first-out" principle and ensures thread security. cursorablelinkedlist is an internal cursor) to locate the two-way linked list of the current element, which is non-thread-safe, but can satisfy the concurrent modification to the container. arraylist is a non-thread-safe and convenient container. general steps for using the object pool: Create a pool object factory and inject the factory into the object pool. to retrieve the pool object, call borrowobject. When returning the pool object, call returnobject, clear () is called to destroy the pool object. To destroy the pool object together, close () is called ().
The following is a sequence chart:
Borrowobject:

 

Returnobject:

 

Invalidateobject:

The Apache connection pool tool library common-DBCP is a specific application of common-pool in database access. when you are familiar with common-pool, you can understand common-DBCP very well. it encapsulates existing connection and statment objects into pool objects poolableconnection and poolablepreparedstatement. then, in these pooled objects, they hold a reference to the object pool. When they are closed, they do not actually close the object, but call:
1. _pool.returnObject(this);  
Or:
1. _pool.returnObject(_key,this);  
Put the connection object back in the connection pool.
The corresponding object pool uses the objectpool, and the latter uses the keyedobjectpool. Because a database only corresponds to one connection, the statement for operations will be divided into many types based on different SQL statements. therefore, the cache must be performed multiple times based on different SQL statements.
In Connection Pool Management, common-DBCP mainly uses two types of objects:
One is poolingdriver and the other is poolingdatasource. The difference between the two is that poolingdriver is a lower-level operation class. It holds a list of connection pool mappings, which are generally used to connect to multiple databases in a JVM, the latter is relatively simple. only one connection pool can be held internally, that is, one data source corresponds to one connection pool.
The following is the structure of common-DBCP:

The following is a tool class written after referring to the common-DBCP example to obtain the connection from the connection pool.

1 ./**
2. * create a connection
3 .*
4. * @ since 2009-1-22 02:58:35 pm
5 .*/
6. Public class connectionutils {
7. // some common-DBCP protocols defined internally
8. Private Static final string pool_driver_key = "JDBC: Apache: commons: DBCP :";
9. Private Static final string polling_driver = "org. Apache. commons. DBCP. poolingdriver ";
10.
11 ./**
12. * obtain the pooled drive
13 .*
14. * @ return
15. * @ throws classnotfoundexception
16. * @ throws sqlexception
17 .*/
18. Private Static poolingdriver getpooldriver () throws classnotfoundexception,
19. sqlexception {
20. Class. forname (polling_driver );
21. Return (poolingdriver) drivermanager. getdriver (pool_driver_key );
22 .}
23.
24 ./**
25. * destroy all connections
26 .*
27. * @ throws exception
28 .*/
29. Public static void destory () throws exception {
30. poolingdriver driver = getpooldriver ();
31. String [] names = driver. getpoolnames ();
32. For (string name: names ){
33. Driver. getconnectionpool (name). Close ();
34 .}
35 .}
36.
37 ./**
38. * obtain the database connection from the connection pool
39 .*/
40. Public static connection getconnection (tablemetadata table)
41. Throws exception {
42. String key = table. getconnectionkey ();
43.
44. poolingdriver driver = getpooldriver ();
45.
46. objectpool pool = NULL;
47. // if the connection pool is not found, an exception is thrown. Catch the exception.
48. Try {
49. Pool = driver. getconnectionpool (key );
50.} catch (exception e ){
51 .}
52.
53. If (pool = NULL ){
54. // build a connection factory based on the database type
55. connectionfactory = NULL;
56. If (table. getdbaddr ()! = NULL
57. & tablemetadata. db_type_mysql = table. getdbtype ()){
58. Class. forname (tablemetadata. mysql_driver );
59. connectionfactory = new drivermanagerconnectionfactory (Table
60. getdburl (), null );
61.} else {
62. Class. forname (tablemetadata. oracle_driver );
63. connectionfactory = new drivermanagerconnectionfactory (Table
64. getdburl (), table. getdbuser (), table. getdbpass ());
65 .}
66.
67. // construct the connection pool
68. objectpool connectionpool = new genericobjectpool (null );
69. New poolableconnectionfactory (connectionfactory, connectionpool,
70. null, null, false, true );
71.
72. // register the connection pool to the driver
73. Driver. registerpool (Key, connectionpool );
74 .}
75.
76. // get a connection from the connection pool
77. Return drivermanager. getconnection (pool_driver_key + key );
78 .}
79.
80 .}