Redis lru Implementation Policy

Redis lru Implementation Policy
When redis is used as the cache, the memory elimination policy determines the memory usage efficiency of redis. In most scenarios, we use LRU (Least Recently Used) as the redis elimination policy. This article introduces the implementation of the redislru policy in a simple way.
What is LRU first? (The following is from Wikipedia)
Discards the least recently used items first. this algorithm requires keeping track of what was used when, which is expensive if one wants to make sure the algorithm always discards the least recently used item. general implementations of this technique require keeping "age bits" for cache-lines and track the "Least Recently Used" cache-linebased on age-bits. in such an implementation, every time a cache-line is used, the age of all other cache-lines changes.
In short, it is to eliminate the least recently used elements each time. The general implementation is to use 'agebits 'For the elements stored in the memory to mark the duration of the element from the last access to the present, so that when LRU is used every time, remove these elements that have not been accessed for the longest time.

Here we first implement a simple LRUCache to facilitate subsequent content understanding. (From leetcod, but here I implemented it again in Python) the following two conditions are met:
1. get (key)-if the element (always positive) exists, move the element to the lru header and return the value of the element; otherwise,-1 is returned.
2. set (key, value)-set the value of a key to value (if this element exists) and move the element to the LRU header. Otherwise, a key is inserted and the value is value. If you check that the cache capacity is exceeded after the key is inserted, delete the least recently used key according to the LRU policy.
Analysis
Here we use a two-way linked list to store elements (k-v key-value pairs) and hash tables to store the corresponding relationship between keys and items. In this way, we can operate keys at O (1) time, while also using the convenience of adding and deleting nodes in DoubleLinkedList. (Get/set can be completed in O (1 ).


Implementation (Python)

 
 

  
  
Class Node:
 
  
  
Key = None

  
  
Value = None

  
  
Pre = None

  
  
Next = None

  
  


  
  
Def _ init _ (self, key, value ):

  
  
Self. key = key

  
  
Self. value = value

  
  


  
  
Class LRUCache:

  
  
Capacity = 0

  
  
Map = {}# key is string, and value is Node object

  
  
Head = None

  
  
End = None

  
  


  
  
Def _ init _ (self, capacity ):

  
  
Self. capacity = capacity

  
  


  
  
Def get (self, key ):

  
  
If key in self. map:

  
  
Node = self. map [key]

  
  
Self. remove (node)

  
  
Self. setHead (node)

  
  
Return node. value

  
  
Else:

  
  
Return-1

  
  


  
  
Def getAllKeys (self ):

  
  
TmpNode = None

  
  
If self. head:

  
  
TmpNode = self. head

  
  
While tmpNode:

  
  
Print (tmpNode. key, tmpNode. value)

  
  
TmpNode = tmpNode. next

  
  


  
  
Def remove (self, n ):

  
  
If n. pre:

  
  
N. pre. next = n. next

  
  
Else:

  
  
Self. head = n. next

  
  


  
  
If n. next:

  
  
N. next. pre = n. pre

  
  
Else:

  
  
Self. end = n. pre

  
  


  
  
Def setHead (self, n ):

  
  
N. next = self. head

  
  
N. pre = None

  
  


  
  
If self. head:

  
  
Self. head. pre = n

  
  


  
  
Self. head = n

  
  


  
  
If not self. end:

  
  
Self. end = self. head

  
  


  
  
Def set (self, key, value ):

  
  
If key in self. map:

  
  
OldNode = self. map [key]

  
  
OldNode. value = value

  
  
Self. remove (oldNode)

  
  
Self. setHead (oldNode)

  
  
Else:

  
  
Node = Node (key, value)

  
  
If len (self. map)> = self. capacity:

  
  
Self. map. pop (self. end. key)

  
  
Self. remove (self. end)

  
  
Self. setHead (node)

  
  
Else:

  
  
Self. setHead (node)

  
  


  
  
Self. map [key] = node

  
  


  
  


  
  
Def main ():

  
  
Cache = LRUCache (100)

  
  


  
  
# D-> c-> B->

  
  
Cache. set ('A', '1 ')

  
  
Cache. set ('B', '2 ')

  
  
Cache. set ('C', 3)

  
  
Cache. set ('D', 4)

  
  


  
  
# Traverse the lru linked list

  
  
Cache. getAllKeys ()

  
  


  
  
# Modify ('A', '1') ==> ('A', 5) to move the node from the end of LRU to the beginning.

  
  
Cache. set ('A', 5)

  
  
# Change the LRU linked list to a-> d-> c-> B

  
  


  
  
Cache. getAllKeys ()

  
  
# The node that accesses key = 'C' is the node that is moved from the LRU Header

  
  
Cache. get ('C ')

  
  
# Change the LRU linked list to c-> a-> d-> B

  
  
Cache. getAllKeys ()

  
  


  
  
If _ name _ = '_ main __':

  
  
Main ()
 
 

Through the simple introduction and implementation above, we have basically understood what is LRU. Let's take a look at the implementation details of the LRU Algorithm in redis, and under what circumstances it will bring problems. Inside redis, the global structure struct redisServer is used to save information related to redis startup, such:

 
 

  
  
struct redisServer {
 
  
  
pid_t pid; /* Main process pid. */

  
  
char *configfile; /* Absolute config file path, or NULL */

  
  
…..

  
  
unsigned lruclock:LRU_BITS; /* Clock for LRU eviction */

  
  
...

  
  
}; 
 
 

RedisServer contains the basic information (PID, configuration file path, serverCron running frequency hz, etc.) after the redis server is started, externally callable module information, network information, RDB/AOF information, log information, copy information, and so on. We can see that in the above struct, lruclock: LRU_BITS stores the lru clock after the server is started, which is a global lru clock. The clock is updated once every 100 ms (you can adjust it by hz. The default value is hz = 10, so a scheduled task is executed every 1000 ms/10 = ms.

Next, let's take a look at the specific implementation of the LRU clock:

 
 

  
  
Server. lruclock = getLRUClock ();
 
  
  
The getLRUClock function is as follows:

  
  
# Define LRU_CLOCK_RESOLUTION 1000/* LRU clock resolution in MS */

  
  
# Define LRU_BITS 24

  
  
# Define LRU_CLOCK_MAX (1 <  Lru */ 
 
  
  
/* Return the LRU clock, based on the clock resolution. This is a time

  
  
* In a reduced-bits format that can be used to set and check

  
  
* Object-> lru field of redisObject structures .*/

  
  


  
  
Unsigned int getLRUClock (void ){

  
  
Return (mstime ()/LRU_CLOCK_RESOLUTION) & LRU_CLOCK_MAX;

  
  
}
 
 

Therefore, lrulock can be up to (2 ** 24-1)/3600/24 = 194 days. If it exceeds this time, lrulock starts again. For redisserver, server. lrulock indicates a global lrulock, so each redisObject has its own lrulock. In this way, each redisObject can be compared with its own lrulock and Global server. lrulock to determine whether it can be eliminated.
Storage object of the value corresponding to the redis key:

 
 

  
  
typedef struct redisObject {
 
  
  
unsigned type:4;

  
  
unsigned encoding:4;

  
  
unsigned lru:LRU_BITS; /* LRU time (relative to server.lruclock) or

  
  
* LFU data (least significant 8 bits frequency

  
  
* and most significant 16 bits decreas time). */

  
  
int refcount;

  
  
void *ptr;

  
  
} robj
 
 

When Will lru be updated? When you access this key, the lru will be updated, so that the key can be promptly moved to the lru header to avoid elimination from the lru. The following is the implementation of this part:

 
 

  
  
/* Low level key lookup API, not actually called directly from commands
 
  
  
* implementations that should instead rely on lookupKeyRead(),

  
  
* lookupKeyWrite() and lookupKeyReadWithFlags(). */

  
  
robj *lookupKey(redisDb *db, robj *key, int flags) {

  
  
dictEntry *de = dictFind(db->dict,key->ptr);

  
  
if (de) {

  
  
robj *val = dictGetVal(de);

  
  


  
  
/* Update the access time for the ageing algorithm.

  
  
* Don't do it if we have a saving child, as this will trigger

  
  
* a copy on write madness. */

  
  
if (server.rdb_child_pid == -1 &&

  
  
server.aof_child_pid == -1 &&

  
  
!(flags & LOOKUP_NOTOUCH))

  
  
{

  
  
if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {

  
  
unsigned long ldt = val->lru >> 8;

  
  
unsigned long counter = LFULogIncr(val->lru & 255);

  
  
val->lru = (ldt << 8) | counter;

  
  
} else {

  
  
val->lru = LRU_CLOCK();

  
  
}

  
  
}

  
  
return val;

  
  
} else {

  
  
return NULL;

  
  
}

  
  
} 
 
 

Next, let's analyze how the key's lru elimination policy is implemented:

 
 

  
  
# Maxmemory policy: how Redis will select what to remove when maxmemory
 
  
  
# Is reached. You can select among five behaviors:

  
  
#

  
  
# Volatile-lru-> Evict using approximated LRU among the keys with an expire set. // use an approximate lru elimination policy in the key with an expiration time set

  
  
# Allkeys-lru-> Evict any key using approximated LRU. // All keys use an approximate lru elimination policy

  
  
# Volatile-lfu-> Evict using approximated LFU among the keys with an expire set. // use the lfu elimination policy in the key with an expiration time set.

  
  
# Allkeys-lfu-> Evict any key using approximated LFU. // All keys use the lfu elimination policy

  
  
# Volatile-random-> Remove a random key among the ones with an expire set. // In the key with an expiration time set, use the random elimination policy.

  
  
# Allkeys-random-> Remove a random key, any key. // All keys use a random elimination policy.

  
  
# Volatile-ttl-> Remove the key with the nearest expire time (minor TTL) // use the ttl elimination policy

  
  
# Noeviction-> Don't evict anything, just return an error on write operations. // do not allow elimination. When a write operation occurs but the memory is insufficient, an error will be returned.

  
  
#

  
  
# LRU means Least Recently Used

  
  
# LFU means Least Frequently Used

  
  
#

  
  
# Both LRU, LFU and volatile-ttl are implemented using approximated

  
  
# Randomized algorithms.
 
 

Here, we will not discuss LFU, TTL elimination algorithm, and noeviction. We will only discuss the specific implementation of the elimination policy in all lru scenarios. (LFU and TTL will be analyzed in detail in the next article ).
LRU elimination scenario: 1. Active elimination. 1.1 Use the scheduled task serverCron to regularly clear expired keys. 2. Passive elimination 2.1 each time the key is written, the memory is insufficient, and activeExpireCycle is called to release some memory. 2.2 each time you access the related key, if it finds that the key has expired, the memory related to the key is directly released.

First, let's analyze LRU active elimination scenarios:ServerCron calls the databasesCron method at intervals of 1000/hz ms to detect and remove expired keys.

   
   

    
    
void databasesCron(void){
 
    
    
/* Expire keys by random sampling. Not required for slaves

    
    
* as master will synthesize DELs for us. */

    
    
if (server.active_expire_enabled && server.masterhost == NULL)

    
    
activeExpireCycle(ACTIVE_EXPIRE_CYCLE_SLOW);

    
    
…..

    
    
} 
   
   

Active elimination is implemented through activeExpireCycle. The logic of this part is as follows:
1. Traverse up to 16 databases. [Defined by macro CRON_DBS_PER_CALL, 16 by default] 2. Randomly select 20 keys with expiration time. [Defined by the macro ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP. The default value is 20.] 3. If the key expires, the memory related to the key is released or put into the invalid queue. 4. If the operation time exceeds the permitted time limit, it can be up to 25 ms. (Timelimit = 1000000 * ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server. hz/100, ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC = 25, server. hz is 10 by default), The elimination operation ends and returns 5. 5. If more than five keys (ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP/4 = 5) in the database are invalid, enter 2. Otherwise, select the next database and enter 2 again. 6. The traversal is complete and the process ends.
The flowchart is as follows:
Note: In the figure above, if the value is greater than or equal to % 5, it may actually expire. Instead, the key value greater than or equal to % 25 should actually expire. Iteration ++ adds 1 at a time when traversing 20 keys ).
Passive elimination-insufficient memory. Call activeExpireCycle to release this step as follows:

   
   

    
    
Logic of the processCommand function on the memory elimination policy:
 
    
    
/* Handle the maxmemory directive.

    
    
*

    
    
* First we try to free some memory if possible (if there are volatile

    
    
* Keys in the dataset). If there are not the only thing we can do

    
    
* Is returning an error .*/

    
    
If (server. maxmemory ){

    
    
Int retval = freeMemoryIfNeeded ();

    
    
/* FreeMemoryIfNeeded may flush slave output buffers. This may result

    
    
* Into a slave, that may be the active client, to be freed .*/

    
    
If (server. current_client = NULL) return C_ERR;

    
    


    
    
/* It was impossible to free enough memory, and the command the client

    
    
* Is trying to execute is denied during OOM conditions? Error .*/

    
    
If (c-> cmd-> flags & CMD_DENYOOM) & retval = C_ERR ){

    
    
FlagTransaction (c );

    
    
AddReply (c, shared. oomerr );

    
    
Return C_ OK;

    
    
}

    
    
}
   
   

Before each command is executed, freeMemoryIfNeeded is called to check the memory and release the corresponding memory. If the memory is not enough after the release, OOM is directly returned to the requesting client. The procedure is as follows:
1. Get the memory mem_reported currently used by the redis server. 2. If mem_reported <server. maxmemory, OK is returned. Otherwise mem_used = mem_reported, go to step 3. 3. traverse the Server Load balancer of the redis instance. mem_used minus the ClientOutputBuffer occupied by all Server Load balancer instances. 4. If AOF is configured, mem_used minus the space occupied by AOF. Sdslen (server. aof_buf) + aofRewriteBufferSize (). 5. If mem_used <server. maxmemory, Return OK. Otherwise, go to Step 6. 6. If the memory policy is set to noeviction, an error is returned. Otherwise, enter 7. 7. if the LRU policy is used and the LRU of VOLATILE is used, maxmemory_samples (5 by default) keys are randomly sampled from the invalidated data set each time, select the key with the largest idletime value for elimination. Otherwise, if it is ALLKEYS_LRU, sampling is performed from the global data. maxmemory_samples (5 by default) keys are randomly sampled each time, and the key with the maximum idletime is selected for elimination. 8. If the memory size exceeds the value of server. maxmemory after it is released, it will be eliminated until the remaining memory size is less than that of server. maxmemory after it is released.
Passive elimination-each access key expires. If the key expires, the memory related to the key is directly released: expireIfNeeded is called to determine whether the key expires. If the key expires, then release and return null; otherwise, return the key value.

Conclusion 1. redis uses the LRU policy as a cache to eliminate data. If too much data expires at the same time, it will take a long time (up to 250 ms) for redis to initiate an active detection ), the maximum application timeout is greater than or equal to 250 ms.

   
   

    
    
Timelimit = 1000000 * ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC/server. hz/100

    
    
ACTIVE_EXPIRE_CYCLE_SLOW_TIME_PERC = 25

    
    
Server. hz> = 1 (10 by default)

    
    
Timelimit <= 250 ms
   
   

2. If the memory usage is too high, it will lead to insufficient memory and initiate a passive elimination policy, so that the application access times out. 3. Adjust the hz parameter reasonably to control the frequency of active elimination each time, so as to effectively alleviate the above timeout problem caused by too many expired keys.