"Redis" Spring Boot leverages Redis's keyspace notifications for message notification

Objective

Requirement: When a key in Redis fails, the value of the failure is written to the database.

Github:https://github.com/vergilyn/redissamples

1, modify the redis.conf

The Redis service installed by default is: Notify-keyspace-events "", modified to notify-keyspace-events Ex;

Location: Redis Install the redis.windows-service.conf or redis.windows.conf. (depending on which configuration the Redis service loads, it looks like Redis 2.8+ is supported)

The corresponding description can be found in the redis.conf

# K-    key space notification to [email protected]<db>__ Prefix # E-    key event notification to [email protected]<  db>__ to prefix # g    del, Expipre, rename and other types unrelated General command notifications, ... # $    string Command # l    List command # s< C11/>set Command # H    hash command # z    ordered set command # x    expiration event (generated each time key expires) # E    Eviction event (generated when key is cleared when memory is full) # a    g$lshzxe alias , so "AKE" means all the events

2, through the jedispubsub realization

Omit the spring boot configuration, see GitHub for the full code.

/** * Key expiration event pushed to topic only key, no value, because once expired, value does not exist. */@Component Public classJedisexpiredlistenerextendsjedispubsub {/** refer to "EVENT NOTIFICATION" in redis.conf under Redis directory, redis default db{0, 15} altogether 16 databases * K Keyspace events, published with [     Email protected]<db>__ prefix.     * E keyevent events, published with [email protected]<db>__ prefix. *     */     Public Final StaticString Listener_pattern = "[Email protected]*__:expired";/**
     * Although it is possible to inject, it seems jedis unusable in Listener-class (unable to establish a connection to Redis), exception message:
* "Only (P) SUBSCRIBE/(p) unsubscribe/quit allowed in this context" */@AutowiredPrivateJedis Jedis;/** * Initialize the process of subscribing by expression * /@Override Public voidOnpsubscribe (String pattern,intSubscribedchannels) {System.out.print ("Onpsubscribe >>"); System.out.println (String.Format ("pattern:%s, Subscribedchannels:%d", pattern, subscribedchannels)); }/** * Processing after a message subscribed by an expression * /@Override Public voidOnpmessage (string pattern, string channel, String message) {System.out.print ("onpmessage >>"); System.out.println (String.Format ("key:%s, pattern:%s, Channel:%s", message, pattern, channel)); }/** * Processing after getting the message of the subscription * /@Override Public voidOnMessage (String channel, String message) {Super. OnMessage (channel, message); }/** * Processing when initializing a subscription * /@Override Public voidOnsubscribe (String Channel,intSubscribedchannels) {Super. Onsubscribe (channel, subscribedchannels); }/** * Cancellation of the subscription process * *@Override Public voidOnunsubscribe (String Channel,intSubscribedchannels) {Super. Onunsubscribe (channel, subscribedchannels); }/** * Cancel the process of subscribing by expression * /@Override Public voidOnpunsubscribe (String pattern,intSubscribedchannels) {Super. Onpunsubscribe (pattern, subscribedchannels); }}

@RunWith (Springrunner.class) @SpringBootTest (classes=jedisexpiredapplication.class) Public classjedisexpiredapplicationtest {@AutowiredPrivateJedis Jedis; @AutowiredPrivateJedisexpiredlistener Expiredlistener; @Before Public voidBefore ()throwsException {jedis.flushall (); Jedis.set (Jedisconfig.defaule_key, "123321"); System.out.println (Jedisconfig.defaule_key + " = "+ jedis.get (Jedisconfig.defaule_key)); System.out.println ("Set expired 5s");    Jedis.expire (jedisconfig.defaule_key,5); } @Test Public voidTestpsubscribe () {/ * Psubscribe is a blocking method that will be blocked until you unsubscribe from the channel, and only if you cancel the subscription will execute the following other code * can onmessage/onpmessage inside receive the message, call the UNSUBSCRI Be ()/onpunsubscribe (); To cancel the subscription so that it executes the following other code * /Jedis.psubscribe (Expiredlistener,jedisexpiredlistener.listener_pattern);//other code}}

Output Result:

Vkey = 123321
Set expired 5s
Onpsubscribe >> pattern: [Email protected]*__:expired, subscribedchannels:1
Onpmessage >> Key:vkey, pattern: [Email protected]*__:expired, channel: [Email protected]__:expired

3. Add MessageListener by implementation

Omit the Redis configuration for spring boot.

@SpringBootApplication Public classRedisexpiredapplicationImplementscommandlinerunner{@AutowiredPrivateRedistemplate redistemplate; @AutowiredPrivateRedisexpiredlistener Expiredlistener;/** * Solve redistemplate key/value garbled problem: * <br/> <a href= "http://www.zhimengzhe.com/shujuku/other/192111 . html ">http://www.zhimengzhe.com/shujuku/other/192111.html</a> * <br/> <a href=" http:/ blog.csdn.net/tianyaleixiaowu/article/details/70595073 ">http://blog.csdn.net/tianyaleixiaowu/article/ details/70595073</a> * @return * *@Bean ("Redis") @Primary PublicRedistemplate redistemplate () {redisserializer<string> Stringserializer =NewStringredisserializer ();        Redistemplate.setkeyserializer (Stringserializer);        Redistemplate.setvalueserializer (Stringserializer);        Redistemplate.sethashkeyserializer (Stringserializer); Redistemplate.sethashvalueserializer (Stringserializer);returnRedistemplate; } @Bean PublicRedismessagelistenercontainer Listenercontainer (redisconnectionfactory redisconnection, Executor Executor) {RedisMe Ssagelistenercontainer container =NewRedismessagelistenercontainer ();//Set up a Redis connection factoryContainer.setconnectionfactory (redisconnection);//Set up the thread pool used by the listener//Container.settaskexecutor (executor);        //Set topic:patterntopic/channeltopic for monitoringTopic Topic =NewPatterntopic (Redisexpiredlistener.listener_pattern);//Set listenerContainer.addmessagelistener (NewRedisexpiredlistener (), topic);returnContainer } @Bean PublicExecutor Executor () {Threadpooltaskexecutor Executor =NewThreadpooltaskexecutor ();        Executor.setcorepoolsize (10);        Executor.setmaxpoolsize (20);        Executor.setqueuecapacity (100);        Executor.setkeepaliveseconds (60); Executor.setthreadnameprefix ("V-thread");//Rejection-policy: How to handle a new task when the pool has reached max size        ///Caller_runs: Does not perform tasks in new threads, but is performed by the thread on which the caller residesExecutor.setrejectedexecutionhandler (NewThreadpoolexecutor.callerrunspolicy ()); Executor.initialize ();returnExecutor } Public Static voidMain (string[] args) {Springapplication.run (redisexpiredapplication.class, args); } @Override Public voidRun (String ... strings)throwsException {Redistemplate.opsforvalue (). Set ("vkey", "Vergilyn", 5, Timeunit.seconds); System.out.println ("Init:set vkey Vergilyn ex 5"); System.out.println ("Thread sleep:10s");        Thread.Sleep (10 * 1000); System.out.println ("thread Recover:get vkey ="+ redistemplate.opsforvalue (). Get ("vkey")); }}

@ComponentPublicclassimplements messagelistener {public    final Static String Listener_pattern = "__key*__:*";    /**     * The client listens to the topic of the subscription and, when there is a message, triggers the method;     * Cannot get value, only get key.     * It is understood that the Redis service will not be able to get the redis-key corresponding redis-value in Java if it fails to notify the Java service when a key fails (or expires).     

* Solution: * Create Copy/shadow key, such as set vkey "Vergilyn"; corresponding Copykey:set Copykey:vkey "" Ex 10;     * The real key is "vkey" (used in business), the fail-trigger key is "Copykey:vkey" (its value is null character in order to reduce memory space consumption). * When the "Copykey:vkey" trigger fails, the value from "Vkey" is invalidated, and after the logic is processed "del vkey" * Defect: * 1: There is an extra key;     (Copykey/shadowkey) * 2: not rigorous, assuming that copykey at 12:00:00 failure, the notification received at 12:10:00, the interval of 10min within the program to modify the key, the resulting is not the value of failure. * (The 1th effect is not big; the 2nd seemingly redis itself pub/sub is not rigorous, after the expiration of the value of the modification, should be in the design/logic to eliminate) * when the "Copykey:vkey" trigger failure, from "vkey" to get the value of the failure, and after the logic processing After "del vkey" * * /@Override Public voidOnMessage (Message message,byte[] bytes) {byte[] BODY = Message.getbody ();//Suggested use: ValueSerializer        byte[] Channel = Message.getchannel (); System.out.print ("onMessage >>" ); System.out.println (String.Format ("Channel:%s, Body:%s, Bytes:%s"                ,NewString (channel),NewString (body),NewString (bytes)); }}

Output Result:

Init:set Vkey Vergilyn ex 5
Thread sleep:10s
OnMessage >> channel: [Email protected]__:expired, Body:vkey, Bytes: __key*__:*
Thread Recover:get vkey = null

4. Questions

1) No matter whether it is jedispubsub or messagelistener, it is impossible to get value.

Personal understanding: In 12:00:00,java push to Redis a command "set Vkey Vergilyn ex 10". The Redis service is now fully aware of the expiration time of this key, and the Redis service expires "Vkey" at 12:00:10.

Then notify Java (that is, callback to Jedispubsub/messagelistener), it is not possible in Java with "Vkey" to get its value.

(The simplest test, breaking points in listener, and then viewing through the Redis-cli.exe command, "Vkey" no longer exists, but listener only enters the message () method)

2) Redis's expire is not strictly instant execution

Excerpted from http://redisdoc.com/topic/notification.html

Redis uses the following two ways to remove expired keys:

• When a key is accessed, the program checks the key, and if the key has expired, the key is deleted.
• The underlying system incrementally finds and removes expired keys in the background to handle keys that have expired but are not accessed.

Redis generates a notification when the expiration key is discovered by any one of the above two programs, and the key is removed from the database expired .

Redis does not guarantee that the time-to-live (TTL) becomes 0 a key that is immediately deleted: If the program does not have access to this expiration key, or if there are many keys with a time-to-live key, then the lifetime of the key is changed 0 until the key is actually deleted, and there may be a significant interval between them.

Therefore, the time that Redis generates notifications is when the expired expiration key is deleted, not when the key's time to live becomes 0 .

If the business cannot tolerate time intervals from expiration to deletion, then it is only in other ways.

3) How to get the value of expire key in the expire callback

Reference: Https://stackoverflow.com/questions/26406303/redis-key-expire-notification-with-jedis

Set vkey somevalueset Shadowkey:vkey "" Ex 10

Equivalent to each key has a corresponding Shadowkey, "Shadowkey" is only used to set the expire time, "key" to save value and participate in business logic.

So when the "Shadowkey" fails notification to listener, the program can get its value through "key" and "Del key" when the logic is processed.

(The value of "Shadowkey" is null or an empty string to save memory space.) )

defects:

1. A lot of invalid shadowkey are superfluous;

2. Data is not rigorous. Assuming that the copykey expires at 12:00:00, the notification is received at 12:10:00, the interval of 10min within the program to modify the key, the resulting is not the value of failure.

Relatively speaking, the 1th is irrelevant, but temporarily more auxiliary key, but will be cleaned out by the program itself, no longer maintenance, or always exist in the Redis cache.

2nd, more is the design logic is defective, you can set the failure time longer, to ensure that in "that interval" can not appear in the failure key modification.

4) Special

Excerpted from http://blog.csdn.net/gqtcgq/article/details/50808729

Redis's publish/subscribe is now out-of-the-box (fire and forget) mode, so reliable notification of events is not possible. That is, if a published/subscribed client is disconnected and then re-connected, all events are lost during the client's broken chain.

Future plans support reliable notification of events, but this can be done by making the subscription and publishing functionality inherently more reliable, or by listening to the subscription and publication of the message in a LUA script, enabling operations like pushing events into a list.

Reference: Time-to-live or expiration time for Redis key expire notification with Jedis

(The following articles are all about the same)

Java implementation of Redis timeout failure key monitoring trigger

Spring boot-uses Redis's keyspace notifications to implement timed task queue Redis timeout failure key monitoring trigger

Redis key Space Notification (Keyspace notifications)

"Redis" Spring Boot leverages Redis's keyspace notifications for message notification