An analysis of the problem of Redis using timeout time

Recently using Redis, the key to do the expiration time, encountered a problem, here is not to say, I set the expiration of a key of 100 days, the results of the test process and there is no problem, but the line is frequently error.

The component uses a Spring-data-redis&jedis.

Jedis.exceptions.JedisConnectionException:Unknown Reply:3
Org.springframework.data.redis.RedisConnectionFailureException:Unknown Reply:3; Nested exception is Redis.clients.jedis.exceptions.JedisConnectionException:Unknown Reply:3

At Org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert (Jedisexceptionconverter.java : 47)

At Org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert (Jedisexceptionconverter.java : 36)

At Org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate ( PASSTHROUGHEXCEPTIONTRANSLATIONSTRATEGY.JAVA:37)

At Org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate ( FALLBACKEXCEPTIONTRANSLATIONSTRATEGY.JAVA:37)

At Org.springframework.data.redis.connection.jedis.JedisConnection.convertJedisAccessException ( jedisconnection.java:181)

At Org.springframework.data.redis.connection.jedis.JedisConnection.expire (jedisconnection.java:773)

At Org.springframework.data.redis.core.redistemplate$7.doinredis (redistemplate.java:648)

At Org.springframework.data.redis.core.redistemplate$7.doinredis (redistemplate.java:641)

At Org.springframework.data.redis.core.RedisTemplate.execute (redistemplate.java:190)

At Org.springframework.data.redis.core.RedisTemplate.execute (redistemplate.java:152)

At Org.springframework.data.redis.core.RedisTemplate.expire (redistemplate.java:641)

......

caused By:redis.clients.jedis.exceptions.JedisConnectionException:Unknown reply:2

At Redis.clients.jedis.Protocol.process (protocol.java:128)

At Redis.clients.jedis.Protocol.read (protocol.java:187)

At Redis.clients.jedis.Connection.getIntegerReply (connection.java:201)

At Redis.clients.jedis.BinaryJedis.expire (binaryjedis.java:330)

At Org.springframework.data.redis.connection.jedis.JedisConnection.expire (jedisconnection.java:771)

... More

Look at this mistake, a little inexplicable, search a bit, found that a lot of is said to be a single case of thread insecurity. Let's use the thread pool, but I'm using spring's encapsulation here, and it's actually using the threading pool.

It was so uncertain that I decided to follow the code and see what the reason was.

First, according to the error prompts, you will be aware of the problem when performing expire, as follows:

At Org.springframework.data.redis.core.RedisTemplate.expire (redistemplate.java:641)
That is, in addition to your own code's last line of error messages, this error message allows me to locate the error because of the execution of the set expiration time on a key.

Next look at the specific code:

Public Boolean expire (K-key, final long timeout, final timeunit unit) {
Final byte[] Rawkey = Rawkey (key);

Final Long rawtimeout = Timeoututils.tomillis (timeout, unit);

Return Execute (New rediscallback<boolean> () {

Public Boolean Doinredis (redisconnection connection) {

try {

Return Connection.pexpire (Rawkey, rawtimeout);

catch (Exception e) {

Driver May is not support pexpire or we'll running on Redis 2.4

Return Connection.expire (Rawkey, Timeoututils.toseconds (timeout, unit));

}

}

}, True);

}

Here is actually a bit of my attention, this note is really,

First of all, he is using the Pexpire command to execute, rather than the expire command we want, if the exception is caught or, it is executed with the expire command. The note here is written if the driver does not support the Pexpire command, or if it is a 2.4 version of Redis, the expire will be executed. First I confirmed that the driver I was using was supported by this command.

Then the officer net, according to the official order introduction, Pexpire is redis2.6 only began to support. Later look at the version, first look at the code inside the Pexpire.

(In fact, according to the following line of stack error information can be known, in fact, the actual implementation of the catch inside the line)

At Org.springframework.data.redis.core.redistemplate$7.doinredis (redistemplate.java:648)
Let's take a look at what the Pexpire has done that might result in throwing an exception.

Public Boolean Pexpire (byte[] key, long Millis) {

/*

* @see DATAREDIS-286 to avoid overflow in Jedis

*

* TODO Remove This workaround while we upgrade to a Jedis version that contains a

* Fix for:https://github.com/xetorthio/jedis/pull/575

*/

if (Millis > Integer.max_value) {

Return Pexpireat (Key, Time () + Millis);

}

try {

if (ispipelined ()) {

Pipeline (New Jedisresult (Pipeline.pexpire) (key, (int) millis), Jedisconverters.longtoboolean ());

return null;

}

if (isqueueing ()) {

Transaction (new Jedisresult (Transaction.pexpire) (key, (int) millis), Jedisconverters.longtoboolean ());

return null;

}

Return Jedisconverters.toboolean (Jedis.pexpire (key, (int) millis));

catch (Exception ex) {

Throw Convertjedisaccessexception (ex);

}

}

This todo is really annoying me, why still like this AH. Here because we want to use the Pexpire command, so the unit converted to milliseconds, 100 days is 8640000000,integer.max_value is 2 31 times minus 1, is 2147483647, obviously, much larger. So instead of executing the pexpireat command, this command sets the specific expiration time based on the Unix time. So it calls the Redis Server command, time to get the current system times, and then adds the required expiration time of 8640000000 is the final expiration time.

Time here is a pit, and if you're using Twemproxy, this is a distributed solution before the Redis cluster occurs.

The wiki in the official code base has a list of supported commands:

Https://raw.githubusercontent.com/twitter/twemproxy/master/notes/redis.md

It can be found from here that it does not support the time command. (The relevant commands can be viewed via Redis official website)

But obviously, the stack does not have the time command error information, so it is necessary to normally execute the Pexpireat method, where the direct execution of the jedis.pexpireat, if the version is less than 2.6, then do not support this command.

Public Boolean pexpireat (byte[] key, long Unixtimeinmillis) {
try {

if (ispipelined ()) {

Pipeline (New Jedisresult (Pipeline.pexpireat (Key, Unixtimeinmillis), Jedisconverters.longtoboolean ());

return null;

}

if (isqueueing ()) {

Transaction (New Jedisresult (Transaction.pexpireat (Key, Unixtimeinmillis), Jedisconverters.longtoboolean ());

return null;

}

Return Jedisconverters.toboolean (Jedis.pexpireat (Key, Unixtimeinmillis));

catch (Exception ex) {

Throw Convertjedisaccessexception (ex);

}

}

Throws an exception, is caught by the outermost layer, uses the expire command to handle.

Come back to see the expire method.

Public Boolean expire (byte[] key, long seconds) {

/*

* @see DATAREDIS-286 to avoid overflow in Jedis

*

* TODO Remove This workaround while we upgrade to a Jedis version that contains a

* Fix for:https://github.com/xetorthio/jedis/pull/575

*/

if (seconds > Integer.max_value) {

Return Pexpireat (Key, Time () + TimeUnit.SECONDS.toMillis (SECONDS));

}

try {

if (ispipelined ()) {

Pipeline (New Jedisresult (Pipeline.expire) (key, (int) seconds), Jedisconverters.longtoboolean ());

return null;

}

if (isqueueing ()) {

Transaction (new Jedisresult (Transaction.expire) (key, (int) seconds), Jedisconverters.longtoboolean ());

return null;

}

Return Jedisconverters.toboolean (Jedis.expire (key, (int) seconds));

catch (Exception ex) {

Throw Convertjedisaccessexception (ex);

}

}

Dizzy, why are you here again? Oh, it's OK, it's seconds, it's not that big. Execute the following Jedis.expire method directly. The stack information also lets you know that the line is actually executing.

At Org.springframework.data.redis.connection.jedis.JedisConnection.expire (jedisconnection.java:771)
It's not a problem this time. Look at the bottom stack information on the web.

Public Long expire (final byte[] key, final int seconds) {
Checkisinmulti ();

Client.expire (key, seconds);

return client.getintegerreply ();

}

Continue to look at the stack information.

At Redis.clients.jedis.BinaryJedis.expire (binaryjedis.java:330)
Then the contrast code is executed by the client.getintegerreply ();

Public Long getintegerreply () {
Flush ();

pipelinedcommands–;

Return (Long) protocol.read (InputStream);

}

Look here, it is Protocol.read (InputStream), there is a problem, follow in, there is a process (is);

private static Object process (final redisinputstream is) {
try {

byte B = is.readbyte ();

if (b = = Minus_byte) {

Processerror (IS);

else if (b = = Asterisk_byte) {

Return processmultibulkreply (IS);

else if (b = = Colon_byte) {

Return Processinteger (IS);

else if (b = = Dollar_byte) {

Return processbulkreply (IS);

else if (b = = Plus_byte) {

Return processstatuscodereply (IS);

} else {

throw new Jedisconnectionexception ("Unknown reply:" + (char) b);

}

catch (IOException e) {

throw new Jedisconnectionexception (e);

}

return null;

}

Find the root cause. It turns out that the B here returns not a few constants that are declared here. According to the error message printed out 2, compared to ASCII can know, 2 is the body start. Well, this is the content of the transport return protocol. If the beginning of the agreement is not right.

What do you mean wrong? Come here, look here, the official introduction of the agreement.

Http://redis.io/topics/protocol

It's used to be resp.

RESP Protocol Description
The RESP protocol is introduced in Redis 1.2, but it became the standard way for talking with the Redis server in Redis 2 .0. This is the protocol your should implement in your Redis client.

RESP is actually a serialization protocol that supports the following data types:simple Strings, Errors, integers, Bulk S Trings and Arrays.

The way RESP is used into Redis as a request-response protocol is the following:

Clients send commands to a Redis server as a RESP Array of Bulk Strings.

The server replies with one of the RESP types according to the command implementation.

In RESP, the type of some data depends on the:

For simple Strings the ' reply is ' + '

For Errors the ' reply is '-'

for integers the ' reply is ': '

For Bulk Strings the ' the reply is ' $ '

For Arrays the ' reply is ' * '

Additionally RESP is able to represent a Null value using a special variation of Bulk Strings or Array as specified.

In RESP different parts of the protocol are always terminated with "\ r \ n" (CRLF).

Different situations will have different protocols at the beginning. Here is not specifically introduced everyone is interested in looking down.

In fact, the interaction with the Redis server is in the exchange of protocol information, nonsense, but if you can not understand the corresponding content of the server, then the error.

Later, the program was changed to reduce the cache time to 20 days, the conversion of the number of milliseconds to 1728000000 is less than 2147483647, so you can perform pexpire instead of pexpireat. Stop the error on the line, the normal execution. Calculate the 2147483647 maximum support time is 24.8 days. So let's look at the set.

Finally only changed the time to solve, but in fact why in the expire place unknow reply not very good confirmation.

and lead to the implementation of the catch in the expire, this reason can be confirmed that the implementation of the pexpireat caused by the time to small after the implementation of the Pexpire, then no longer error. The only difference between the two is the difference between the time, that is, the if judgment, suspicion pexpireat and Pexpire there (after all, this problem was submitted in March 2014, suspected that the line jar does not have this section of the If judgment results in an error).

Here's a summary of the two points, one we found by checking the problem, the time command can not be used in the use of the Twemproxy agent, here through the spring code can be found that if the setting of too long is bound to encounter this.

The other is that we have learned the Redis communication protocol.