Redis Cache invalidation Mechanism

Redis cache failure story to start with the expire command, expire allows the user to specify a time-out for a key, and the value corresponding to the key will be erased after that time. This article mainly on the basis of the analysis of Redis source code to stand on the Redis designer's perspective to consider the problem of Redis cache failure.

Redis cache invalidation mechanism

The Redis cache invalidation mechanism is designed to address a very common scenario for caching applications, speaking of a scenario:

In order to reduce the pressure on the back-end database, we are happy to use the Redis service to put the frequency of the change is not very high from the DB load into the cache, so for a period of time we can directly from the cache data, but we hope that after a period of time, we re-from the DB Load out the current data into the cache, how does this matter?

The question is raised, how to solve this problem? Well, we're very familiar with the language tools at hand, and we believe we can quickly write a logic like this: We record the last time we load data from DB, and then each time we respond to the service, we have to decide whether the time is out of date or to reload from the db .... Of course this method is also possible, but when we look at Redis command document, we find that we did what we didn't need to do, and Redis itself provides this mechanism, and we can do it easily with the help of the expire command:

EXPIRE Key 30

The above command is the key set 30 seconds to expire, more than this time, we should not access this value, so far we probably understand what is the cache invalidation mechanism and caching failure mechanism of some scenarios, then we continue to delve into this problem, How is the Redis cache invalidation mechanism implemented?

Delay failure mechanism

The delay failure mechanism is that when a client requests a key, Redis checks the key for the client request operation, and if the key expires to handle it, the delay mechanism is also called the negative failure mechanism. Let's look at the service end-side execution stack facing GET request processing under the T_string component:

GetCommand

Getgenericcommand

lookupkeyreadorreply

Lookupkeyread

expireifneeded

The key point is that the Expireifneed,redis Key's get operation will determine if the value associated with the key is invalidated, insert an episode first, and let's look at where the value is actually stored in Redis:

typedef struct redisdb {

dict *dict; /* The keyspace for this DB * /

Dict *expires; /* timeout of keys with a Timeout Set */

dict *blocking_keys; /* Keys with clients waiting for data (blpop) * /

dict *ready_keys; /* Blocked keys that received a PUSH */ /c20>

dict *watched_keys; /* Watched keys for MULTI/EXEC CAS * /

int ID;

long long avg_ttl; /* Average TTL, just for stats * /

} redisdb;

Above is a structure defined in Redis, Dict is a redis implementation of a dictionary, that is, each db will include the above five fields, we here only care about two dictionaries, one is dict, one is expires:

1. Dict is used to store normal data, such as we execute the set key "hahaha", this data is stored in the Dict.

2. Expires is used to store keys associated with the expiration time, such as the expire key 1 that we have executed on top of it, this time adding a record to the expires.

Looking back at the expireifneeded process, the following:

1. Find out the expiration time of key from expires, if there is no indication that the corresponding key does not set the expiration time, return directly.

2. If it is a slave machine, then return directly, because Redis to ensure data consistency and implementation is simple, the cache invalidation of the initiative to the Master machine, slave machine does not have the privilege to invalidate the key.

3. If the master machine is currently in place and key expires, Master will do two important things: 1) write the delete command to the AoF file. 2) Notify Slave that the current key is invalid and can be deleted.

4. Master deletes the value for the key from the local dictionary.

Active failure Mechanism

Active failure mechanism is also called the active failure mechanism, that is, the service end of the timer to check the failed cache, if the failure of the corresponding operation.

We all know that Redis is single-threaded, event-driven, and a eventloop,eventloop in Redis is responsible for handling two types of events:

1. A class is an IO event, which is isolated from the underlying multiplexer.

2. A class is a timed event, which is used primarily to perform timed events on a task.

It seems that Redis's EventLoop and Netty as well as JavaScript's eventloop function design probably similar, on the one hand the network I/O event processing, on the one hand can do some small tasks.

Why is the single-threaded model of Redis, because the active failure mechanism logic of Redis is executed as a scheduled task from the main thread, the related code is as follows:

If(aecreatetimeevent(server. El, 1, servercron, NULL , NULL) = = ae_err) {

redispanic("Can ' t create the Servercron time event.") );

Exit(1);

}

Servercron is the function pointer for this timed task, Adcreatetimeevent registers the Servercron task with the EventLoop and sets the initial execution time to 1 milliseconds. Next, everything we want to know is in the Servercron. Servercron do a bit more, we only care about the content related to this article, that is, how the cache invalidation is implemented, I think the code to do what things, call stack is more intuitive:

Aeprocessevents

processtimeevents

Servercron

Databasescron

activeexpirecycle

Activeexpirecycletryexpire

EventLoop through the processing of timed tasks, triggering the execution of Servercron logic, the final implementation of key expiration processing logic, it is worth mentioning that the activeexpirecycle logic can only be done by master.

Legacy Issues

The processing mechanism of Redis to cache invalidation is probably divided into two kinds, one is negative processing when the client accesses key, one is the main thread periodically actively executes the cache invalidation cleanup logic, the above article has not introduced to some details yet, but for the topic of the Redis cache invalidation implementation mechanism, This article leaves a few questions:

1. Why does Redis cache fail logic work only with Master?

2. It is mentioned above that if the client accesses Slave,slave and does not clean the failed cache, then does this client get a stale cache?

3. What are the pros and cons of the two cache invalidation mechanisms described above? Why are Redis designers so designed?

4. Server-to-client request processing is single-threaded, single-threaded and to deal with the failed cache, will it affect the service capabilities of Redis itself?

Redis Cache invalidation Mechanism