Cache penetration, Cache breakdown, Cache avalanche Solution Analysis __redis

Preface

To design a caching system, the problem that has to be considered is: cache penetration, cache breakdown and failure of avalanche effect.
Cache Penetration

Cache penetration means querying a data that does not exist, because the cache is written passively when it is not hit, and for fault tolerance, if no data is found from the storage layer, the cache is not written, which causes the nonexistent data to be queried at the storage level for each request, losing the meaning of the cache. When the traffic is large, the db is probably dead, and if someone uses a nonexistent key to attack our application frequently, this is the loophole. Solution

There are a number of ways to effectively address cache penetration problems, the most common is the use of a bitmap filter, all the possible data hashes into a large enough, a certain nonexistent data will be intercepted by this bitmap, thereby avoiding the underlying storage system query pressure. There is also a much simpler and more brutal approach (which we are using), if the data returned by a query is empty (whether the data does not exist or the system fails), we still cache the empty result, but it will expire in a short period of up to five minutes. Cache Avalanche

A cache avalanche is when we set the cache with the same expiration time, causing the delay to fail at the same time, and all requests are forwarded to the db,db instantaneous pressure avalanche. Solution

The avalanche effect on the underlying system is terrible when the cache fails. Most system designers consider using a lock or queue to ensure that a single thread (process) of the cache is written to avoid the failure of a large number of concurrent requests falling onto the underlying storage system. Here to share a simple solution when the cache failure time is dispersed, such as we can be in the original failure time based on a random value, such as 1-5 minutes of random, so that each cache expiration rate of recurrence will be reduced, it is difficult to trigger the event of collective failure. Cache Breakdown

For some keys that have their expiration time set, it is a very "hot" data if they may be accessed at some point in time with high concurrency. This time, you need to consider a problem: The cache is "breakdown" problem, and the difference between the cache avalanche is here for a key cache, the former is a lot of key.

When the cache expires at a point in time, there is a large number of concurrent requests for the key at this point in time, and these requests find that cache expiration typically loads data from the backend DB and is set back to the cache, when a large concurrent request can overwhelm the backend db. Solution 1. Use Mutex (mutex key)

The industry's more common practice is to use mutexes. Simply put, when the cache fails (the value taken out is null), instead of going to load db immediately, use some of the caching tool's operations (such as Redis's Setnx or Memcache's add) to set a mutex key first, When the operation returns successful, the load DB operation is performed and the cache is reset, otherwise the whole get cache method is retried.

SETNX is the abbreviation for "set if not exists", which is only set when it does not exist, and can be used to achieve the effect of the lock. The version does not implement SETNX expiration time before redis2.6.1, so here are two versions of the Code reference:

2.6.1 Pre-stand-alone version lock
string get (String key) {  
   String value = Redis.get (key);  
   if (value  = = null) {  
    if (redis.setnx (Key_mutex, "1")) {  
        //3 min timeout to avoid mutex holder  
        crash . Expire (Key_mutex, 3 *)  
        value = Db.get (key);  
        Redis.set (key, value);  
        Redis.delete (Key_mutex);  
    } else {  
        //other threads rest 50 milliseconds before retrying  
        Thread.Sleep (m);  
        Get (key);}}  

Latest Version Code:

public string Get (key) {
      String value = Redis.get (key);
      if (value = = null) {//= timeout for cache value expiration
          //Set 3min, prevent del operation from failing, the next cache expiration cannot load DB
		  if (Redis.setnx (Key_mutex, 1, 3 * 60) = = 1) {  //representative Set success
               value = Db.get (key);
                      Redis.set (key, value, expire_secs);
                      Redis.del (Key_mutex);
              } else {  //This time to represent the same time other threads have load db and back to the cache, and then try to get the cache value can sleep
                      ();
                      Get (key);  Retry
              }
          } else {return
              value      
          }
 }

Memcache Code:

if (memcache.get (key) = = NULL) {  
    //3 min timeout to avoid mutex holder crash  
    if Memcache.add (Key_mutex, 3 * 60 * 1000 = = = True) {  
        value = Db.get (key);  
        Memcache.set (key, value);  
        Memcache.delete (Key_mutex);  
    } else {sleep  
        (m);  
        Retry ();  
    }  

2. "In advance" using mutex (Mutex key):

Set 1 timeout values inside value (TIMEOUT1), TIMEOUT1 is smaller than the actual memcache timeout (timeout2). When read from cache to TIMEOUT1 found that it has expired, immediately extend the TIMEOUT1 and reset to the cache. The data is then loaded from the database and set to the cache. Pseudo code is as follows:

v = memcache.get (key);  
if (v = = null) {  
    if (Memcache.add (Key_mutex, 3 * 1000) = = True) {  
        value = Db.get (key);  
        Memcache.set (key, value);  
        Memcache.delete (Key_mutex);  
    } else {sleep  
        (m);  
        Retry ();  
    }  
else {  
    if (v.timeout <= now ()) {  
        if (Memcache.add (Key_mutex, 3 * * 1000) = = True) {  
            //Extend the TIM Eout for the other threads  
            V.timeout + = 3 * * 1000;  
            Memcache.set (Key, V, key_timeout * 2);  
  
            Load the latest value from db  
            v = db.get (key);  
            V.timeout = key_timeout;  
            Memcache.set (key, Value, Key_timeout * 2);  
            Memcache.delete (Key_mutex);  
        } else {sleep  
            (m);  
            Retry ();  
        }  
      

3. "Never Expires":

The word "never Expires" here contains two layers of meaning:

(1) from the Redis, it does not set the expiration time, which is guaranteed, will not appear hot key expiration problem, that is, "physics" is not the period.

(2) From the functional perspective, if not expired, it is not static. So we put the expiration time exists in the value of the key, if found to expire, through a background asynchronous thread for the construction of the cache, that is, "logic" expired

In practice, this approach is very good for performance, the only problem is that the rest of the thread (not building the cached thread) may be accessing old data, but this is tolerable for general internet features.

String get (Final String key) {  
        V v = redis.get (key);  
        String value = V.getvalue ();  
        Long timeout = V.gettimeout ();  
        if (V.timeout <= system.currenttimemillis ()) {  
            //asynchronous Update background exception execution  
            Threadpool.execute (new Runnable () {  
                public void Run () {  
                    String Keymutex = "Mutex:" + key;  
                    if (Redis.setnx (Keymutex, "1")) {  
                        //3 min timeout to avoid mutex holder crash  
                        (Redis.expire, 3 *) Keymutex 1/>string Dbvalue = Db.get (key);  
                        Redis.set (key, dbvalue);  
                        Redis.delete (Keymutex);}}  
            );  
        return value;  
}

4. Resource protection:

Using the hystrix of Netflix, the main thread pool can be used to isolate and protect the resource, if the application to the cache is built.

Four solutions: No best and only the most appropriate

Solution	Advantages	Disadvantages
Simple distributed mutex (mutex key)	1. Simple Thinking 2. Ensure consistency	1. Increased complexity of code 2. Risk of deadlock 3. There is a risk of thread pool blocking
"Advance" use mutexes	1. Ensure consistency	Ditto
But period (this article)	1. Build the cache asynchronously without blocking the thread pool	1. No guarantee of consistency. 2. Code complexity increases (each value maintains a timekey). 3. Occupy a certain amount of memory space (each value should maintain a timekey).
Resource Isolation Component Hystrix (article)	1. Hystrix technology is mature, effectively guarantee the back end. 2. Hystrix monitoring is powerful.	1. Partial access has a demotion policy.

Four network of program sources, details please link: http://carlosfu.iteye.com/blog/2269687?hmsr=toutiao.io&utm_medium=toutiao.io&utm_source =toutiao.io

Summary

For business systems, is always the specific situation of specific analysis, there is no best, only the most appropriate.

Finally, for caching system common cache full and data loss problem, need according to specific business analysis, usually we adopt LRU strategy to deal with overflow, Redis Rdb and aof persistence strategy to ensure data security under certain circumstances.