Commons-pool parsing

First throws a classic question:

Are aware that most applications that connect to MySQL use frameworks such as C3P0, DBCP Proxool, and so on to manage database connection pooling. Database connection pooling is undoubtedly a long-connected approach. So why does the MySQL classic eight-hour problem arise?

I began to doubt that since it is a long connection there must be constant heartbeat detection mechanism has been constantly harassing the service side, then how can the server also detect a eight-hour non-static connection? In fact, this is the improper use of the API (or the API implementation should set some default values to prevent small white people with a lot of problems)

Our common Jedis-xxx.jar and C3p0xx.jar are implemented using Commons-pool as the object pool. These upper-level jar applications generally need to implement a xxxpoolableobjectfactory within the Commons-pool to do the management of the specific pool objects (typical Factory mode). The following is a brief example of Jedis:

1 How objects are created

Jedis's jedisfactory inherits the org.apache.commons.pool.BasePoolableObjectFactory implementation of the Makeobject () method:

  PublicObject Makeobject ()throwsException {FinalJedis Jedis =NewJedis ( This. Host, This. Port, This. Timeout);        Start the Socket connection jedis.connect (); if(NULL!= This. Password) {Jedis.auth ( This. Password); }        if(Database! = 0) {jedis.select (database); }                returnJedis; }

This code is very understood, is nothing more than the bottom of the socket to start connecting to the corresponding server, such as the lowest code:

  Public voidConnect () {if(!isconnected ()) {            Try{Socket=NewSocket (); //@wjw_addSocket.setreuseaddress (true); Socket.setkeepalive (true);//Would monitor the TCP connection is validSocket.settcpnodelay (true);//Socket buffer whetherclosed, to ensure timely delivery of dataSocket.setsolinger (true, 0);//Control calls Close () method, the underlying socket is closed immediately//<[email Protected]_addSocket.connect (Newinetsocketaddress (host, port), timeout);                Socket.setsotimeout (timeout); OutputStream=NewRedisoutputstream (Socket.getoutputstream ()); InputStream=NewRedisinputstream (Socket.getinputstream ()); } Catch(IOException ex) {Throw NewJedisconnectionexception (ex); }        }    }

2

Apache Commons pool, commonly used to implement various connection pools, such as database connection pool, Redis connection pool, etc.

The following is an example of a car rental company that illustrates the basic way Commons Pool works:

Genericobjectpool (Car rental company)

As a car rental company, two services are required to provide car rental and return vehicles, and it also manages all those vehicles, needs to buy new cars as the business grows, requires the destruction of vehicles that have not been safely driven, and regularly conducts safety checks on vehicles.

Pooledobject (all vehicles for car rental companies)

The car rental company's vehicles are divided into three categories: idle rental vehicles (idle Objects), loaned vehicles (Active Objects), vehicles considered discarded (anandoned Objects)

Borrow Object (car rental)

• A1: The world is so big, a young man wants to rent a car to hang out.
• A2: The boss first see if there is a free car
• A3.1: If yes, the recently returned car is loaned out and marked as loaned (Active), if there is no free car, buy a car, but also marked as loaned (this is a company not bad money)
• A3.2: The boss rents the marked car to the young

Return Object (return the car)

• B1: The world is so big, the young people finally finished shopping, returned to return the car
• B2: The boss puts the car back in the parking lot and changes the mark to idle (idle) and can be rented by someone else.

Testonborrow/testonreturn (check out when renting/returning)

This company not only not bad money, it is also very responsible for the safety of vehicles, for rent out of the car, whether it is taken from the free vehicle, or the new buy back, will first check the quality of the car, always can not hang the young, if found to have problems, immediately change another. Return, will also check again, if there is a problem, throw away (true local tyrants), in addition, the company also specially invited a vehicle security officer, regularly idle for a period of time the vehicle for safety testing (evict Thread), a problem also thrown away.

It seems that everything is fine.

In reality, however, there are always unexpected occurrences:

When the young man borrowed the car, he found the world was getting bigger and more reluctant to go home. Security Inspector regular inspection found that the car was borrowed for half a year, has not returned, is not lost? So took out the cell phone, "pa" pressed a bit, remote car out of the fire, marked as scrap vehicle (abandoned), as scrap treatment.

Evict Thread (regular inspection of security personnel)

• C1: For vehicles that have been returned marked as free, the security inspectors regularly check them, if not used for more than a period of time, to see if it is broken, bad will be in time to void (C2).
• D1: For objects marked as loaned, the security inspector periodically checks that the loan has not yet been retired for a long time, and is directly void (D2).

Well, the story is finished, I hope everyone to Commons pool understand.

Interested students can continue to look at the problem we have encountered:

We use Jedis as a Redis client operation, and in the context of pressure testing, we occasionally find Jedis reported this exception:ClassCastException - [B cannot be cast to java.lang.Long

Google Baidu on the internet, found that most of the netizens said it was due to pipeline operation, abnormal connection did not properly destory off, and directly put back into the connection pool, the next thread was taken, take to the connection residual pipeline operation results, resulting in a type conversion error.

This explanation sounds very sensible, but repeatedly check the code, found that the encapsulation of the exception is done, and the problem is not used pipeline operation, should not be the case of netizens said.

So suspicion is not the connection pool out of the problem, multiple threads to the same connection did a different operation, get the wrong data caused, but the famous commons-pool appeared such a low-level error, it is impossible?

After flipping through the Commons-pool code, it turns out that it might be the ghost of the regular check-up security officer who said that.

For the loaned object, we configured to loan out after more than 10 seconds do not return is void, theoretically for the operation of Redis, 10 seconds is indeed enough, but we do a further encapsulation of jedispool, in some special cases, there will be a holding connection for more than 10 seconds (this does not expand) , causing the connection to be used by the program, while reading the data processing of Redis, the clean thread is destroyed by the innocent (called jedis.quit() ),

The return value of Jedis's quit command is a byte array, and our operation returns a long, so an ClassCastException - [B cannot be cast to java.lang.Long exception occurs.

The final solution is to increase the definition of obsolete time appropriately.

Commons-pool parsing