About Codis for the original Redis expiration event solution

1. RedisHow to handle outdated events1.1. RedisProcessing ExpiredKeyWay

Redis KeyThere are two ways to expire: passive mode and active way
whenClientsAn attempt was made to access a setting that has an expiration time and has expiredKeyis the active expiration mode.
but it is not enough to think that there may be someKeyWill never be accessed again, these set the expiration time of theKeyIt also needs to be deleted after it expires. Therefore,Rediswill periodically follow thea batch of machine tests set the expiration timeKeyand to be processed. Tested to an expiredKeywill be deleted. The typical way is, Redisdo it every second.Tenthe following steps:
1.Random Test -setting an expiration time .Key
2.Delete all discovered expiredKey
3.If the deletedKeymore than -Repeat steps for each of the1

This is a proactive approach

1.2. RedisHandling Expired Events

@Test          public void testredisevent ()  {                      jedis jedis = newjedis ("192.168.92.134",  8000);                    jedis.psubscribe (NewJedisPubSub ( )  {                             public voidonpmessage (String  pattern, string channel,                                                  Stringmessage)  {                                       System.err.println (pattern);                                       SYSTEM.ERR.PRINTLN (channel);                                       SYSTEM.ERR.PRINTLN (message);                              }                    }, "[email protected]*__:expired");          }

when the listener expires Key when the Redis Add this Key through pub/sub to send the event.

because Codis does not support commands that contain pub/sub, This listener event is no longer supported by Codis.

2. Solution Solutions2.1. AdoptJavaof theDelayqueueSolve

Delayqueue Description: An unbounded blocking queue for the Delayed element from which the element can be extracted only when the delay expires. The head of the queue is the Delayed element with the longest save time after the delay expires. If the delay has not expired, the queue does not have a header, and poll returns null. An expiration occurs when an element's Getdelay (timeunit.nanoseconds) method returns a value that is less than or equal to 0. These elements are not treated as normal elements even if you cannot use take or poll to remove elements that are not expired. For example, thesize method returns the count of both the expired and the unexpired elements. This queue does not allow the use of NULL elements.

The code is as follows:

public class redisevent<k, v> implements runnable{     Public delayqueue<delayitem<pair<k, v>>> q = new delayqueue <DelayItem<Pair<K, V>>> ();     public void run ()  {         while  (True) {             DelayItem<Pair<K, V>> delayItem = null;             try {                 delayitem = q.take ();                 if  (delayitem !=  null)  {                     //  Timeout Object Handling                      pair<k, v> pair = delayitem.getitem ();                      System.out.println ("key : "  + pair.first +  ", value : "  +  pair.second);                 }            } catch  ( Interruptedexception e)  {                 e.printstacktrace ();             }        }    }     Public static void main (STRING[]&NBsp;args)  throws InterruptedException {         Redisevent<string, string> re = new redisevent<string, string> ();         //Expired key,value         Pair<string, string> pair1 = new pair<string, string> ("Key1",  " Value1 ")         //2 sec Expiration          delayitem<pair<string,string>> d1 = new delayitem<pair<string, String>> (Pair1, timeunit.seconds.tonanos (2));         Re.q.add (D1);         system.out.println (Re.q.poll ());         thread.sleep (1000 * 1);         system.out.println (Re.q.poll ()); &NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;//2 seconds to get data          Thread.Sleep (1000 * 2);         delayitem<pair<string, String>> delayitem = re.q.poll ();         System.out.println (Delayitem);        pair<string, string>  pair =delayitem.getitem ();         system.out.println ("Key  :  " + pair.first + ", value :  " + pair.second);         system.out.println (Re.q.poll ());     }

Output Result:

Nullnull[email Protected]key:key1, Value:value1null

Advantages and disadvantages of this scenario:

Advantages: Self-realization, easy to use, easy to modify, expand

disadvantage: When there are too many expired events and the expiration time is too long, you need to Delayqueue Too much storage Key , Consumption JVM Memory

2.2. use of third-partyQueueQueue Implementation

need to Queue Support Delay Job , we are now aware of: BEANSTALKD , Sidekiq

BEANSTALKD Introduction:

Beanstalk , a high-performance, lightweight, distributed memory queuing system originally designed to reduce high capacity by performing time-consuming tasks asynchronously in the background Web Application System's page access delay, support over 9.5 million User's Facebook causes application.

BEANSTALKD Core Concepts within the design:

Job

A task that requires asynchronous processing is the basic unit in BEANSTALKD, which needs to be placed in a tube.

Tube

A well-known task queue, used to store a uniform type of job, is an object of producer and consumer operations.

Producer

Job producer, put a job into a tube with a put command.

Consumer

Job consumers, through the reserve/release/bury/delete command to obtain job or change the status of the job.

BEANSTALKD The life cycle of a job in 2 is shown in. A job has ready, RESERVED, DELAYED, buried four states. When producer puts a job directly, the job is in the ready state, waits for consumer to process it, and if the delay Put,job is selected first to delayed state, the wait time is migrated to ready state. After consumer acquires the current ready job, the job's status is migrated to reserved so that the other consumer can no longer manipulate the job. When consumer completes the job, it can choose Delete, release or Bury operation; After delete, the job is extinct from the system and can no longer be acquired; the release operation re-migrates the job state back to Ready (Also The State migration operation can be deferred, so that other consumer can continue to acquire and execute the job; Interestingly, the Bury operation can hibernate the job, wait until it is needed, and then kick the dormant job back to the ready state. You can also deleteburied a status job. It is this interesting operation and state that can be based on this to make a lot of meaning of the application, such as the implementation of a circular queue, you can reserved the status of the job to hibernate, and so on without the ready state of the job, and then the buried state of the job once kick back to the ready state.

Pros: No need to implement it yourself, use a third party queue

Disadvantages:

1. beanstalkd currently does not provide a master-slave synchronization + failover mechanism, which can be a single point of risk in the application. In a real-world application, you can use a database to provide persistent storage for a job.

2. need to spend cost learning third party queue

2.3. using a database implementation

The expired key,value is placed in the database, and the database is scanned using thread timing.

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/73/7E/wKioL1X_s0_R1jckAABNNCwQlJw022.jpg "title=" 1.png " alt= "Wkiol1x_s0_r1jckaabnncwqljw022.jpg"/>

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/73/80/wKiom1X_sSDC4k2AAADEGTMz3lY646.jpg "title=" 2.png " alt= "Wkiom1x_ssdc4k2aaadegtmz3ly646.jpg"/>

Start Three threads:

thread 1: Scan for events that have a status of delay and expiration timebased on expiration time , change the status to Ready

thread 2: Handle An event with a status of ready and change the ready state to done

thread 3: Delete The event of the done State

Pros: Data persistence without loss of event data

Cons: Thread timed scan, expired event has deferred processing

2.4. UseRedisImplement(Recommended)

because Codis does not support pub/sub, re-add the Redis service, which is used only for expired event handling

Pros: The implementation is the same as the original, no need to modify any code

disadvantage: Need to add machine to do Redis service, master and standby need to use keepalive and other processing

This article is from the Java Architect's Road blog, so be sure to keep this source http://eric100.blog.51cto.com/2535573/1696775

About Codis for the original Redis expiration event solution