In-depth understanding of. NET MemoryCache

Tag: This bin res is the limit INF.

Summary

MemoryCache is a memory cache class that is available at the beginning of the. Net Framework 4.0, which makes it easy to cache data within a program and easily manage the validity of the data. This type enables similar functionality for the cache classes commonly used in ASP. And can be adapted to richer usage scenarios. In the use of memorycache often have a variety of questions, how the data organized? Is it possible to use more efficient organization and usage? How is the data timeout controlled? In order to know its reason, this article has carried on the thorough analysis to the memorycache principle and the realization way, at the same time has learned many industry mature component's design idea in the analysis process, has opened the more open idea for the future work

This article is intended for the. NET 4.5.1 version, which is slightly different in the subsequent. NET version of MemoryCache, Welcome to add

Article content is longer, estimated reading time about 1 hours

The MemoryCache class inherits from the ObjectCache abstract class and implements the IEnumerable and IDisposable interface. Similar functions are implemented with the cache class commonly used by ASP, but the memorycache is more generic. You do not have to rely on a class library when using it System.Web , and you can use MemoryCache to create multiple instances in the same process.

When using memorycache, there are often questions about how internal data is organized in this class. How is the timeout for cache entries handled? Why does it advertise that it is thread-safe? To answer these questions, the next step is to explore the internal implementation of MemoryCache with reference source.

MemoryCache Internal data structure

Within the MemoryCache class, data is organized in relation to the three classes of Memorycachestore, Memorycachekey, and Memorycacheentry, respectively:

• Memorycachestore: Hosting Data
• Memorycachekey: Constructing a retrieval item
• Memorycacheentry: Caching the true representation of internal data

The relationship is roughly as follows:

As can be seen intuitively from the diagram, a MemoryCache instance object can contain multiple Memorycachestore objects, depending on the hardware environment in which the program resides, and the number of CPUs. Inside the MemoryCache, the Memorycachestore object is like a small database, carrying a variety of data. Therefore, to understand the internal data structure of memorycache, we need to understand the position and function of Memorycachestore first.

Memorycachestore

The type is a container inside memorycache that is really used to host data. It directly manages the program's memory cache entries, and since the data is being hosted, there must be some attributes in that type that are related to the data store. The specific behavior is that there is a private property of type MemoryCache HashTable _entries , in which all the cache entries it manages are stored.

Hashtable _entries = new Hashtable(new MemoryCacheEqualityComparer());

When it comes time to get data in MemoryCache, MemoryCache's first step is to find the Memorycachestore object that stores the key being looked for, rather than just going to a Dictionary type or HashTable The type of object that is directly looking for the result.

The way to find Memorycachestore in MemoryCache is also interesting, the main logic in the MemoryCache GetStore method, the source code is as follows (to understand the convenience of adding some comments):

internal MemoryCacheStore GetStore(MemoryCacheKey cacheKey) {    int hashCode = cacheKey.Hash;//获取key有关的hashCode值    if (hashCode < 0) {        //避免出现负数        hashCode = (hashCode == Int32.MinValue) ? 0 : -hashCode;    }    int idx = hashCode & _storeMask;    //_storeMask跟CPU的数目一致，通过&进行按位与计算获取到对应的Store    //本处代码是.NET 4.5的样子，在.NET Framework 4.7.2版本已经改成了使用%进行取余计算，对于正整数来说实际结果是一样的。    return _storeRefs[idx].Target;}

Since there may be multiple Memorycachestore objects, there is a need for rules to determine what is stored in each store. As can be seen from the source code, MemoryCache uses the CPU's kernel as a mask and uses the mask and Key's hashcode to compute the cache entry's attribution, which is really simple and efficient.

Memorycachekey

The class function of Memorycachekey is relatively simple, it is mainly used for encapsulating key of cache item and related common method.

The initialization in Memorycachestore is mentioned above _entries , and the parameter of the constructor is a Memorycacheequalitycomparer object, what is this and what is the function of it?

The Memorycacheequalitycomparer class implements the IEqualityComparer interface, which defines the method for determining the equality of values in the hash table to analyze the following source code:

internal class MemoryCacheEqualityComparer: IEqualityComparer {    bool IEqualityComparer.Equals(Object x, Object y) {        Dbg.Assert(x != null && x is MemoryCacheKey);        Dbg.Assert(y != null && y is MemoryCacheKey);        MemoryCacheKey a, b;        a = (MemoryCacheKey)x;        b = (MemoryCacheKey)y;        //MemoryCacheKey的Key属性就是我们在获取和设置缓存时使用的key值        return (String.Compare(a.Key, b.Key, StringComparison.Ordinal) == 0);    }    int IEqualityComparer.GetHashCode(Object obj) {        MemoryCacheKey cacheKey = (MemoryCacheKey) obj;        return cacheKey.Hash;    }}

As can be seen from the code, the real function of Memorycacheequalitycomparer is to define the Memorycachekey comparison method. The key attribute in Memorycachekey is used to determine whether two two memorycachekey are equal. So when we get and set the relevant content in MemoryCache, we use the results of the relevant operations for Memorycachekey.

Memorycacheentry

This type is the real form of the cache entry in memory. It inherits from the Memorycachekey type, and adds a lot of properties and methods on this basis, such as judging whether to timeout or not.

First look at the overall situation of the class:

In general, the properties and methods in Memorycacheentry are mainly three classes:

1. Content related to the cache, such as key, Value
2. Status related to cached content, such as state, hasexpiration method, etc.
3. Related events related to cache content, such as Callcacheentryremovedcallback method, callnotifyonchanged method, etc.

By understanding how data is organized in MemoryCache, you can help understand how data is being queried from memorycache in a step-by-step manner.

How to query data from Memorycahe

What is the process of getting data from MemoryCache? Overall, there are two broad categories: getting data and validating validity.

The above steps are expressed as a flowchart:

The detailed steps are this:

1. Validation of query parameters Regionname and key for effective judgment
2. Constructs a Memorycachekey object for subsequent steps to query and compare existing data
3. Get Memorycachestore object, narrow query scope
4. Extract the Memorycacheentry object from the Hashtable type attribute of the Memorycachestore, and get the data corresponding to the key
5. Determine the validity of Memorycacheentry objects and perform data validation work
6. Handle Memorycacheentry, such as the sliding timeout time, and other access-related logic

As you can see here, you can't help but think of the design of other cache systems that I've known before, just as history sometimes has amazing similarities, and a well-designed caching system does seem to have a lot of similarities at some point. By learning the good design of others, you can learn a lot of things, such as the next cache timeout mechanism.

MemoryCache timeout mechanism

MemoryCache can select a persistent cache when setting a cache entry or automatically disappear after a time-out. Where the cache policy can choose either a fixed time-out and a sliding timeout (note that only two of these time-out policies can be selected, the following explains why there is such a rule).

The time-out management mechanism for cache entries is a must for caching systems (such as Redis and memcached), with active and passive triggering in Redis, and memcached with a passive trigger check. So how does memory cache MemoryCache internally manage the timeout mechanism for cache entries?

MemoryCache the time-out management mechanism for cache entries is similar to Redis, and there are two types: periodic and lazy.

Delete periodically

Since memorycache internal data is managed in Memorycachestore objects, periodic checks are also likely to be a behavior within the Memorycachestore object.

By carefully reading the source code, we find that the method is called in the constructor of Memorycachestore InitDisposableMembers() , which is as follows:

private void InitDisposableMembers() {    //_insertBlock是MemoryCacheStore的私有属性    //_insertBlock的声明方式是：private ManualResetEvent _insertBlock;    _insertBlock = new ManualResetEvent(true);    //_expires是MemoryCacheStore的私有属性    //_expires的声明方式是：private CacheExpires _expires;    _expires.EnableExpirationTimer(true);}

This attribute is related to the timeout mechanism discussed in this section _expires . Because ". NET reference source" does not have the relevant source code for this cacheexpires class, it is not possible to know the specific implementation way, so find the same name method from the Mono project to explore the specific implementation of this type.

Class cacheexpires:cacheentrycollection{public static TimeSpan Min_update_delta = new TimeSpan (0, 0, 1);    public static TimeSpan Expirations_interval = new TimeSpan (0, 0, 20);    public static Cacheexpireshelper helper = new Cacheexpireshelper ();    Timer timer; Public Cacheexpires (Memorycachestore store): Base (store, helper) {} public new void Add (Memorycacheen Try entry) {entry.        Expiresentryref = new Expiresentryref (); Base.    ADD (entry); } public new void Remove (Memorycacheentry entry) {base.        Remove (entry); Entry.    Expiresentryref = Expiresentryref.invalid; } public void Utcupdate (Memorycacheentry entry, DateTime utcabsexp) {base.        Remove (entry); Entry.        Utcabsexp = Utcabsexp; Base.    ADD (entry);                public void Enableexpirationtimer (bool enable) {if (enable) {if (timer! = null)            Return var period = (int) expirations_interval. TotalMilliseconds;        Timer = new timer ((o) = Flushexpireditems (true), NULL, period, period); } else {timer.            Dispose ();        timer = null; }} public int flushexpireditems (bool blockinsert) {return base.    Flushitems (Datetime.utcnow, cacheentryremovedreason.expired, Blockinsert); }}

Through the source code in mono, it can be seen that a timer is used inside the cacheexpires to trigger a timed check through the timer. The method of the Cacheentrycollection class is used when triggering FlushItems . The implementation of this method is as follows;

protected int FlushItems (DateTime limit, CacheEntryRemovedReason reason, bool blockInsert, int count = int.MaxValue){    var flushedItems = 0;    if (blockInsert)        store.BlockInsert ();    lock (entries) {        foreach (var entry in entries) {            if (helper.GetDateTime (entry) > limit || flushedItems >= count)                break;            flushedItems++;        }        for (var f = 0; f < flushedItems; f++)            store.Remove (entries.Min, null, reason);    }    if (blockInsert)        store.UnblockInsert ();    return flushedItems;}

In FlushItems(***) the logic, by traversing all the cache entries and the time-out period, the found time-out cache entry is removed and the remove operation is cleared to implement periodic deletion of the cached item. It is inferred from the functionality of this class in the Mono project that implementations in the. NET Framework should have similar functionality, i.e. each MemoryCache instance will have a task that is responsible for timing checks, and is responsible for handling all cache entries that are timed out.

Lazy Delete

In addition to the timing of the deletion, MemoryCache also implemented the function of lazy deletion, the implementation of this feature is more simple than the timing of the deletion, and very practical.

What does lazy deletion mean? The simple thing to do is to use the cache entry to determine if the cache entry should be deleted, rather than waiting for a dedicated cleanup task to be cleaned up.

The previous article describes how data is organized in MemoryCache, and since it is the logic that is triggered when it is used, the lazy deletion is bound to be related to how Memorycachestore gets the cache. Take a look at the Get internal logic of its approach:

internal MemoryCacheEntry Get(MemoryCacheKey key) {    MemoryCacheEntry entry = _entries[key] as MemoryCacheEntry;    // 判断是否超时    if (entry != null && entry.UtcAbsExp <= DateTime.UtcNow) {        Remove(key, entry, CacheEntryRemovedReason.Expired);        entry = null;    }    // 更新滑动超时的时间和相关的计数器    UpdateExpAndUsage(entry);    return entry;}

As can be seen from the code, Memorycachestore found the relevant key corresponding to the cache entry, and did not return directly, but first check the cache item time-out. If the cache entry times out, the item is deleted and NULL is returned. This is how lazy deletions are implemented in MemoryCache.

Cache expiration policy for MemoryCache

When you add a cache entry to a MemoryCache instance, you can select three expiration policies:

1. Never timeout
2. Absolute timeout
3. Sliding timeout

The caching policy is specified when the cache entry is added/updated (either by using the Add or Set method), and the cache timeout policy is set by specifying the object when the cache is being manipulated CacheItemPolicy .

The cache time-out policy is not arbitrarily specified, and within MemoryCache CacheItemPolicy there is a built-in check mechanism for the object. First look at the source:

private void Validatepolicy (CacheItemPolicy policy) {//Check expiration time policy combination setting if (policy). Absoluteexpiration! = objectcache.infiniteabsoluteexpiration && policy. SlidingExpiration! = objectcache.noslidingexpiration) {throw new ArgumentException (r.invalid_expiration_combinatio    N, "policy"); }//Check the sliding time-out policy if (policy. slidingexpiration < Objectcache.noslidingexpiration | | Oneyear < policy. slidingexpiration) {throw new ArgumentOutOfRangeException ("policy", RH.    Format (R.argument_out_of_range, "slidingexpiration", Objectcache.noslidingexpiration, oneyear)); }//Check callback set if (policy. RemovedCallback! = NULL && policy.    Updatecallback! = null) {throw new ArgumentException (r.invalid_callback_combination, "policy"); }//Check the priority setting if (policy. Priority! = Cacheitempriority.default && policy. Priority! = cacheitempriority.notremovable) {throw new ArgumentOutOfRangeException ("policy", RH. Format (r.argument_out_of_range, "Priority", Cacheitempriority.default, cacheitempriority.notremovable)); }}

Summarize the logic in the source code, the time-out policy setting has the following several rules:

1. Absolute timeout and sliding timeout cannot exist at the same time (this is the reason that the previous article said that the two choice)
2. Not if the sliding timeout is less than 0 or more than 1 years.
3. RemovedCallbackAnd UpdateCallback cannot be set simultaneously
4. Cached Priority properties cannot be outside the enumeration range (default and Notremovable)

MEMORYCAHCE Thread Safety mechanism

According to the MSDN Description: MemoryCache is thread-safe. Then, when manipulating cache entries in MemoryCache, MemoryCache guarantees that the program behaves atomically, without problems such as data pollution caused by multiple threads working together.

So, how did memorycache do that?

MemoryCache uses a locking mechanism internally to guarantee the atomicity of data item operations. The lock shares the same lock in each memorycachestore, that is, the data inside the same memorycachestore, and does not affect each other memorycachestore.

The following scenarios exist for locking logic:

1. Traversing MemoryCache Cache entries
2. Add/Update cache entries to MemoryCache
3. Perform memorycache destruction
4. To remove a cache entry in a memorycache

Other scenarios are better understood, and it is worth mentioning the implementation of scenario 1 (traversal). In MemoryCache, a lock-and-copy approach is used to handle traversal requirements, ensuring that no exceptions occur during traversal.

The traversal in. NET 4.5.1 is implemented in such a way that:

protected override IEnumerator<KeyValuePair<string, object>> GetEnumerator() {    Dictionary<string, object> h = new Dictionary<string, object>();    if (!IsDisposed) {        foreach (MemoryCacheStore store in _stores) {            store.CopyTo(h);        }    }    return h.GetEnumerator();}

store.CopyTo(h);the implementation is defined in Memorycachestore, that is to say, each store locking unlock is a separate process, reducing the scope of the lock mechanism is also an important means of improving performance. The main logic of the CopyTo method is a simple traversal under the control of the lock mechanism:

internal void CopyTo(IDictionary h) {    lock (_entriesLock) {        if (_disposed == 0) {            foreach (DictionaryEntry e in _entries) {                MemoryCacheKey key = e.Key as MemoryCacheKey;                MemoryCacheEntry entry = e.Value as MemoryCacheEntry;                if (entry.UtcAbsExp > DateTime.UtcNow) {                    h[key.Key] = entry.Value;                }            }        }    }}

Somewhat unexpectedly, when traversing the memorycache, in order to achieve thread safety during traversal, the way to implement is actually to copy the data another. Of course, it is not always true that a copy is a copy, and if the cache entry is a reference type, it is just a pointer. But it seems best to use less, in case the cache is a basic type, once the amount of data is large, the memory pressure during the traversal is not negligible.

Summarize

In this paper, MemoryCache for the organization and use of data for the axis, in-depth analysis of memorycache for some of the daily applications have a direct relevance to the implementation of the function. MemoryCache the data into different hasttable through multiple Memorycachestore objects, and ensures that the operation is thread-safe inside each store using a lock-in method, which also improves the performance of the global lock to some extent. In order to manage the cache entry timeout, MemoryCache takes two different management measures, which effectively guarantees the validity of the cache entry's time-out management and removes the related cache in time after timeout to free memory resources. Through the analysis of these functions, we understand the data structure and data query method inside the MemoryCache, and have mastered many instructive experiences for the future work.

Resources

• MemoryCache class
• MemoryCache class Source
• Optimization practice of charging system based on multilevel cache
• Deep understanding of redis_memcached failure principle
• Cacheexpires Source
• Cacheentrycollection Source
• CacheItemPolicy class
• Thread Safety

In-depth understanding of. NET MemoryCache