[Go] High concurrent access to avoid object cache failure triggering dogpile effect

Avoid the dogpile effect caused by redis/memcached cache invalidation

The Dogpile effect (the cache stampede effect) may occur when the cache fails under redis/memcached high concurrent access.

Recommended reading: Nginx optimization scheme under high concurrency http://www.linuxidc.com/Linux/2013-01/78791.htm

• Avoid the dogpile effect of memcached cache

  Memcached's read-through cache process: The client reads the cache, and the client generates the cache without a message.
  Memcached Cache Example:

  $MC = new Memcached (), $MC->addservers (Array (    ' 127.0.0.1 ', 11211, +),    array (' 127.0.0.1 ', 11212, 30),    Array (' 127.0.0.1 ', 11213, 30)); $data = $MC->get (' Cached_key '); if ($MC->getresultcode () = = = Memcached::res_notfound) {    $data = Generatedata ( ); Long-running process    $mc->set (' Cached_key ', $data, Time () + 30);} Var_dump ($data);

  If the above Generatedata () is an operation or a database operation that takes 3 seconds (or more). When the cache server is unavailable (for example, a cache instance is down, or the network causes) or the cache fails instantaneously, if there is a large number of access requests, There will be a sharp rise in machine CPU consumption or database operations in a short time, which could cause database/web server failures.

  There are usually two ways to avoid such dogpile effects:

  • Using a standalone update process
    Use a separate process (for example, cron Job) to update the cache instead of having the Web server update the data cache on the fly. For example: a data statistic needs to be updated every five minutes (but each time it takes 1 minutes), you can use cron job to calculate this data, and update the cache. In this case, the data will always exist, even if it does not exist and does not have to worry about the dogpile effect, because the client does not update the cached operation. This method is suitable for global data that does not require immediate operation. But it doesn't work well for user objects, lists of friends, comments, and so on.
  • Use "Lock"
    In addition to using a standalone update process, we can also add "lock" to allow only one client request to update the cache at a time to avoid the dogpile effect.
    The process is probably like this:
    1. A requested cache is not hit
    2. A request "lock" the cache key
    3. b requested Cache not hit
    4. b requests need to wait until "lock" is released
    5. A request is complete and the "lock" is released
    6. B Request Cache Hit (due to the operation of a)

    memcached examples of using "locks":

    function Get ($key) {    global $mc;    $data = $MC->get ($key);    Check if cache exists    if ($MC->getresultcode () = = = memcached::res_success) {        return $data;    }    Add Locking    $MC->add (' Lock: '. $key, ' locked ', +);    if ($MC->getresultcode () = = = memcached::res_success) {        $data = Generatedata ();        $MC->set ($key, $data,);    } else {        while (1) {            usleep (500000);            $data = $MC->get ($key);            if ($data!== false) {break                ;            }        }    }    return $data;} $data = Get (' Cached_key '); Var_dump ($data);

    There is a flaw in the above approach, that is, when the cache fails, all requests need to wait for a request to complete the cache update, which will undoubtedly increase the pressure on the server.
    This is relatively good if the cache update can be triggered for a period of time before the data fails, or if the cache fails to return only the appropriate state for the client to handle itself according to the return state.

    The following get method is to return the corresponding state handled by the client:

    Class Cache {Const RES_SUCCESS = 0;    Const GENERATEDATA = 1;    Const NOTFOUND = 2;    Public function __construct ($memcached) {$this->MC = $memcached;        Public function Get ($key) {$data = $this->mc->get ($key); Check if cache exists if ($this->mc->getresultcode () = = = memcached::res_success) {$this->_s            Etresultcode (cache::res_success);        return $data;        }//Add locking $this->mc->add (' Lock: '. $key, ' locked ', 20); if ($this->mc->getresultcode () = = = memcached::res_success) {$this->_setresultcode (cache::generatedata            );        return false;        } $this->_setresultcode (Cache::notfound);    return false;    } Private Function _setresultcode ($code) {$this->code = $code;    } public Function Getresultcode () {return $this->code; The public function set ($key, $data, $expiry) {$this->mc->set ($key, $dATA, $expiry); }} $cache = new cache ($MC), $data = $cache->get (' Cached_key '); switch ($cache->getresultcode ()) {case Cache::res_    SUCCESS://... break;    Case Cache::generatedata://Generate Data ... $cache->set (' Cached_key ', Generatedata (), 30);    Break Case Cache::notfound://not found ... break;}

    When the above memcached cache fails, only one client request will return the Cache::generatedata state, and the others will return Cache::notfound. The client can do the appropriate processing by detecting these states.
    Note that the TTL value for "lock" should be greater than generatedata () consumption time, but should be less than the TTL value of the actual cached object.

  • Avoid the dogpile effect of Redis cache

    Redis Normal Read-through Cache Example:

    $redis = new Redis (), $redis->connect (' 127.0.0.1 ', 6379), $data = $redis->get (' Hot_items '); if ($data = = = False) {
           //calculate hot items from MySQL, says:it takes seconds for this process    $data = Expensive_database_call (); 
           
            //store the data with a minute expiration    $redis->setex ("Hot_items", Max, $data);} Var_dump ($data);
           

    As with the memcached cache, the Dogpile effect can also be triggered when the Redis cache fails in high concurrency.
    Below is an example of how Redis avoids the dogpile effect by using "lock":

    $redis = new Redis (), $redis->connect (' 127.0.0.1 '); $expiry          =///Cached 600S$RECALCULATED_AT = 100s $lock _length     =;  Lock-key expiry 20s$data = $redis->get ("Hot_items"), $ttl  = $redis->get ("Hot_items"); if ($ttl <= $ Recalculated_at && $redis->setnx (' Lock:hot_items ', true)) {    $redis->expire (' Lock:hot_items ', $ Lock_length);    $data = Expensive_database_call ();    $redis->setex (' Hot_items ', $expiry, $data);} Var_dump ($data);

    The process above is this:

    1. Normally gets the cached data of key hot_items and also gets the TTL (the time remaining from expiration)
    2. The above Hot_items expiration time is set to 600s, but when the Hot_items is ttl<=100s, the cached update process is triggered
    3. $redis->setnx(‘lock:hot_items‘, true)Try to create a key as a "lock". If the key already exists, SETNX will not do any action and the return value is false, so only one client will return a true value to the IF statement to update the cache.
    4. Set the 20s expiration time for the key as "lock", in case the PHP process crashes or processing expires, allow another process to update the cache after the key as "lock" expires.
    5. If the Expensive_database_call () is called in the IF statement, the most recent data is saved normally to Hot_items.

[Go] high concurrent access to avoid object cache invalidation caused dogpile effect