Introduction to local application caching algorithms and caching policies

Special statement: The article is I search on the Internet some of the information, a little to tidy up, but also hope that we do not misunderstand, specifically from that I have also forgotten. Also please do not misunderstand!!!

Through the design of good data block, prefetch, replace, update and other algorithms to improve the cache content hit rate and consistency.

1) Data chunking

Memcached by default, a mechanism named slab allocator is used to allocate and manage memory. Prior to the advent of this mechanism, the allocation of memory was performed simply by malloc and free for all records. However, this approach can lead to memory fragmentation, aggravating the burden on the operating system memory manager, and in the worst case, cause the operating system to be slower than the memcached process itself. Slab Allocator was born to solve the problem.

2) prefetch Policy

Most caching algorithms use prefetching policies to put part of the disk data into the cache in advance,to further reduce the diskI/O,increase the cache hit ratio. Predict data segments that may be requested in the future by recording and analyzing past data request Patterns,Place data segments that are more likely to be accessed into the cache.VoDThere is a strong sequential data access pattern in video on demand:that is, the user does not makeVCRoperation of the case,sequentially access all parts of the entire movie file,access to data is rarely accessed again. This type of data access pattern is in the designVoDThe caching algorithm must be considered. So the sequential Prefetch method is suitable forVoDSystem.

3) Replacement Policy

because the data access patterns of different systems are not the same, it is difficult for the same cache policy to achieve satisfactory performance in various data access modes .

researchers propose different caching strategies to suit different needs. Cache policies can be categorized into the following categories :

based on Access time : This algorithm organizes the cache queue according to the access time of each cache entry , and decides to replace the object. such as LRU.

based on frequency of access : This class of algorithms organizes caches with the frequency of access to cached items. such as LFU,LRU-2,2Q,lirs.

access time and frequency : by taking into account the access time and frequency , the caching strategy still has good performance when the data access mode changes. such as FBR,lrfu,alrfu. Most of these algorithms have an adjustable or adaptive parameter , which allows the cache policy to strike a certain balance between access time and frequency based on the parameter's adjustment.

< Span style= "Font-size:small;" > based on access mode : Some applications have more specific data access features , system design A&l cache policy , Span style= "font-family: ' The song Body '; The sarc

< Span style= "Font-size:small;" > access-time-based caching strategy: LRU (leastrecentlyused) is a widely used caching algorithm. The algorithm maintains a cache entry queue , , will be at the end of the queue , That is, delete the last time that was accessed the oldest item , put the new section in the queue header.

but The LRU algorithm only maintains the access time information of the cache block , does not consider factors such as the frequency of access , and does not get the ideal hit rate in some access modes. For example , in a VoD system , where there is no VCR operation , the data is accessed from the pre-to-post sequence , and the data that has been accessed is not accessed again. So the LRU algorithm will finally be replaced with the most recently accessed data not applicable to the VoD system.

Frequency-based Caching policy:

LFU (leastfrequentlyused) sorts the blocks in the cache by the frequency at which each cache block is accessed , replacing one of the least frequently accessed cache queues when the cache space is full . similar to the disadvantages of LRU, LFU maintains only the frequency of access information for each item, and for a cache entry , if the entry has a very high frequency of access in the past and the most recent access frequency is low, When the cache space is full, the item is difficult to replace from the cache , resulting in a drop in hit ratio.

Lru-2[2, 3] The algorithm records the last two times each cached page was accessed. Replace the page with a second-to-last access time that is now the longest.

< Span style= "Font-size:small;" > IRM (independentreferencemodel) access mode , Lru-2 has the best expected hit rate , because lru-2< Span style= "font-family: ' The song Body '; > algorithm to maintain a priority queue , so the complexity of the algorithm is Logn (N)

< Span style= "Font-size:small;" >2q[4] (2 Queues) lru queue replaced lru-2< Span style= "font-family: ' The song Body '; The priority queue , The algorithm time complexity from logn descending to constant , and the amount of space required to maintain the cache entries is more than lru-2 small.

< Span style= "Font-size:small;" >LIRS[5] (Low inter-referencerecency Set) maintain a variable length LRU queue , LRU 2 times Llirs lirs algorithm in irm access mode can get very high hits , sdd access mode efficiency is significantly reduced.

for VoD system , which is based on frequency-of-access strategies, captures hot movie fragments , freeing large requests for hotspot fragments from slow disk I/O. But movie hotspots change over time , and such policies are not able to take advantage of the sequential access patterns of VOD, so it is not suitable for a VOD system to simply replace operations with frequency-of-access metrics .

Balance access time and frequency:

fbr[6 "(Frequency Based replacement) maintain a LRU queue , new " old three paragraphs. Maintains a count value for each cache entry in the queue. When an item in the cache is hit , The hit cache entry is moved to new mru end , If the item was originally located in Old segment , then its count value plus 1,, delete Old end )

< Span style= "Font-size:small;" >LRFU[7] (leastrecently frequentlyused) C (x), change according to the following formula.

< Span style= "Font-size:small;" > in t moment , C (x) =1+2- was accessed to 2-λ C (x): x< Span style= "font-family: ' The song Body '; > When the replace operation is not accessed , C (x) , Lrfu algorithm behaves like LFU; and at that time , lru

LRFU Although the access time and frequency factors are taken into account by a value , the algorithm cannot make corresponding adjustments when the access mode changes due to the fixed value , resulting in degraded performance. Alrfu[8] (Adaptive LRFU) Improved LRFU In this regard . By monitoring the history of the data access mode , ALRFU dynamically adjusts the value to adapt to the change of the data access mode , and shows better adaptability than LRFU.

Access-mode-based caching policy:

< Span style= "Font-size:small;" > literature [9] for VoD system features proposed A&l algorithm. The algorithm estimates the future frequency of access for each cache entry by recording its historical access time and number of accesses. With this frequency value as the measure , Replace the one with the smallest value when the cache is replaced. Because this algorithm considers the vod system data Access features , So it is widely used in vod system.

< Span style= "Font-size:small;" > but a&l The algorithm calculates the total number of accesses since the cache segment was generated directly, Frequency to generate the cache weights , no consideration Vod The video's access hotspots change over time. The cache that affects the new hotspot fragment , can't adapt to the dynamic changes of the movie hotspot.

IBM 's proposed sarc[10] is a caching algorithm for large servers that dynamically adapts to data access patterns with random access and sequential access. Sarc adapts the two different access modes by dividing the random access and sequential access into two separate queues for management. By analyzing the simulation test data curve of cache size-hit ratio, the cost function of replacing random queue with sequential queue item is obtained. When a cache substitution is made , the cost -less queue is replaced by a price function of the two queues.

4) Update policy

The update policy is divided into active and passive updates:

Passive update

Set the time at which the key expires and let it expire automatically.

is the cache data expiration setting that we typically do, such as Redis and Memcache, which provide APIs such as expire to set the expiration time for k-v.

In general, the business can tolerate a period of time (for example, one hours), cached data and real data (such as MySQL, hbase, etc.) inconsistent data (in general, the cache can improve the speed of access to reduce the back-end load), then we can set a certain amount of time the expiration time, after the data expires, Then get the data from the real data source, put it back in the cache, and continue to set the expiration time.

Proactive Updates

Update the cache at the same time as the DB is updated. The general business is used in conjunction with active and passive updates.

The business is highly consistent with data and needs to update the cached data immediately after the real data is updated.

Specific practices: For example, you can use the message system or other means (such as database triggers, or other data sources to complete the listener mechanism) to notify cache updates.

Introduction to local application caching algorithms and caching policies