Programming Technology cache writing (3)

The last time we talked about multi-level cache, this chapter details how to design the memory cache. The last time we talked about multi-level cache, this chapter details how to design the memory cache.

I. Analysis and Design

Assume that a project has a certain concurrency and uses multi-level caching as follows:

Before designing a memory cache, we need to consider the following:

1: Memory and Redis data replacement, as far as possible to improve the data hit rate in the memory, reduce the pressure of the next level.

2: limits the memory capacity. you need to control the cache quantity.

3: If hotspot data is updated differently, you need to configure an expiration time for a single key.

4: A good cache expiration deletion policy.

5: The complexity of the cache data structure is as low as possible.

About replacement and hit rate: We use the LRU algorithm because it is simple and the cache key hit rate is also good.

LRU is used to remove the least recently accessed data, which is frequently accessed as hotspot data.

LRU data structure: the key structure is required because the key priority is increased and the key is eliminated. I have seen that most implementations use the linked list structure,

That is, when new data is inserted to the head of the linked list and hit data is moved to the head. Add complexity O (1) to move and obtain complexity O (N ).

Is there any lower complexity? With a Dictionary, the complexity is O (1), and the performance is the best. So how can we ensure that the cache priority is increased?

II. O (1) LRU implementation

We define an LRUCache Class, construct the maxKeySize parameter to control the maximum number of caches.

Use ConcurrentDictionary as our cache container and ensure thread security.

Public class LRUCache
 
  
: IEnumerable
  
   
> {Private long ageToDiscard = 0; // age start point of elimination private long currentAge = 0; // The latest age of the cache private int maxSize = 0; // maximum cache capacity private readonly ConcurrentDictionary
   
    
Cache; public LRUCache (int maxKeySize) {cache = new ConcurrentDictionary
    
     
(); MaxSize = maxKeySize ;}}
    
   
  
 

The preceding two auto-increment parameters ageToDiscard and currentAge are defined to mark the new and old degrees of each key in the cache list.

The core implementation steps are as follows:

1: Each time a key is added, currentAge automatically increases and the currentAge value is assigned to the Age of this cache value. currentAge always increases.

public void Add(string key, TValue value)       {           Adjust(key);           var result = new TrackValue(this, value);           cache.AddOrUpdate(key, result, (k, o) => result);       }       public class TrackValue       {           public readonly TValue Value;           public long Age;           public TrackValue(LRUCache
 
   lv, TValue tv)           {               Age = Interlocked.Increment(ref lv.currentAge);               Value = tv;           }       }
 

2: If the maximum number is exceeded. Check whether the dictionary contains a key of the ageToDiscard age. If no cyclic auto-increment check exists, the key is deleted or added successfully.

AgeToDiscard + maxSize = currentAge. in this way, the old data can be eliminated under O (1), instead of moving through the linked list.

public void Adjust(string key)        {            while (cache.Count >= maxSize)            {                long ageToDelete = Interlocked.Increment(ref ageToDiscard);                var toDiscard =                      cache.FirstOrDefault(p => p.Value.Age == ageToDelete);                if (toDiscard.Key == null)                    continue;                TrackValue old;                cache.TryRemove(toDiscard.Key, out old);            }        }

Expired deletion policy

In most cases, the LRU algorithm has a high hit rate on hotspot data. However, if a large amount of occasional data access occurs, the memory will store a large amount of cold data, that is, cache pollution.

This will cause LRU to fail to hit hotspot data, resulting in a sharp drop in the cache system hit rate. You can also use variant algorithms such as LRU-K, 2Q, and MQ to increase the hit rate.

Expiration configuration

1: We try to avoid cold data resident memory by setting the maximum expiration time.

2: In most cases, the cache time is inconsistent, so the expiration time of a single key is increased.

Private TimeSpan maxTime; public LRUCache (int maxKeySize, TimeSpan maxExpireTime) {}// TrackValue adds the creation time and expiration time public readonly DateTime CreateTime; public readonly TimeSpan ExpireTime;

Deletion policy

1: it is best to use timed deletion to delete expired keys. In this way, the occupied memory can be released as soon as possible, but it is obvious that a large number of timers cannot afford the CPU.

2: therefore, we use the inert deletion method to check whether the key expires and delete it directly when it expires.

public Tuple
 
   CheckExpire(string key)        {            TrackValue result;            if (cache.TryGetValue(key, out result))            {                var age = DateTime.Now.Subtract(result.CreateTime);                if (age >= maxTime || age >= result.ExpireTime)                {                    TrackValue old;                    cache.TryRemove(key, out old);                    return Tuple.Create(default(TrackValue), false);                }            }            return Tuple.Create(result, true);        }
 

3: Although the performance of the inert deletion is the best, it still does not solve the cache pollution problem for cold data. So we need to clean it regularly.

For example, open a thread and traverse and check the key once every five minutes. This policy can be configured based on actual scenarios.

public void Inspection()        {            foreach (var item in this)            {                CheckExpire(item.Key);            }        }

Inert deletion + regular deletion can basically meet our needs.

Summary

If it continues to improve, it is the prototype of the memory database, similar to redis.

For example, add a notification to delete a key and add more data types. This article also references the implementation of redis and Orleans.