Memcached sliding expiration implementation and binary optimization

JUSFR original, reproduced please indicate from the blog park.

The first part preface

HttpRuntime.Cache.Insert (string key, object value, CacheDependency dependencies, DateTime absoluteexpiration, TimeSpan The slidingexpiration parameter of the SlidingExpiration method ensures that the cache is accessed and the validity time is extended, while Memcached does not implement this capability, as the 3rd party class library enyimcaching provides The overloads provided by the Memcachedclient.store () method are still expired by absolute time, and the former is not the same semantics, the purpose of this article is to deal with this problem and to give a more optimized scheme.

     Public BOOL string Object value);      Public BOOL string Object value, DateTime expiresat);      Public BOOL string object value, TimeSpan validfor);

Enyimcaching interface Encapsulation is good, it should be noted that such as configuration errors, etc. will not cause an exception to throw, so it is best to use a test case to confirm.

[TestMethod] Public voidOnline () {using(Memcachedclient client =NewMemcachedclient ("enyim.com/memcached") ) {String key= Guid.NewGuid (). ToString ("N"); Object value=client.            Get (key);            Assert.isnull (value); varexist = client. Tryget (Key, outvalue);            Assert.isfalse (exist);            Assert.isnull (value); Value=Guid.NewGuid (); Client.            Store (Storemode.set, key, value); exist= client. Tryget (Key, outvalue);            Assert.istrue (exist);        Assert.isnotnull (value); }    }

View Code

It is also important to note that when the client time is inconsistent with the server time of the Memcached, it will cause the expiration time and the expected inconsistency , considering the development under Windows, such as Ubuntu server can use the following command to synchronize time and restart Memcached Process

    sudo  Time . windows.com     sudo hwclock-w        sudo service memcached restart

Then the original sliding expiration test is used

[TestMethod] public void Sliding () {using (Memcachedclient client= New Memcachedclient ("enyim.com/memcached") ) {String key= Guid.NewGuid (). ToString ("N"); Object value=Guid.NewGuid (); Client.            Store (Storemode.set, key, Value, Timespan.fromseconds (15D));            Thread.Sleep (Timespan.fromseconds (10D)); var exist=client.            Tryget (key, out value);             Assert.istrue (exist);            Assert.isnotnull (value);            Thread.Sleep (Timespan.fromseconds (10D)); exist=client.            Tryget (key, out value); Assert.istrue (exist); //failedAssert.isnotnull (value); }    }

The test case failed, the expiration parameter used here is 15 seconds, the cache is checked after 10 seconds, and after 10 seconds the cache has expired.

The second part realizes

You can already see, if you want the 2nd 10 seconds, the cache can still be accessed, then the 1th 10 seconds, the cache has been removed once again. Indeed, the barrier is that when the cache is taken out without the time information, I look through the API, as if there is no way (if the fallacy, please correct), so we can take a generic wraper to save time and value to the cache, when taken out of the comparison.

The definition of slidingcachewraper<tcache> is as follows, note [Serializable] is needed in the use of enyimcaching:

    [Serializable] public    class Slidingcachewraper<TCache> {public        tcache Cache {get; Private set; Public        DateTime Cachingtime {get; private set;}        Public TimeSpan slidingexpiration {get; private set;}        Public Slidingcachewraper (Tcache cache, TimeSpan slidingexpiration) {            = cache;             = slidingexpiration;             = datetime.now;        }    }

The approximate logic is this: when it is necessary to store a sliding expiration, we first use Slidingcachewraper to wrap up this cache entry, record the cache time and sliding expiration parameters, and feed into Memcached, when the item is removed from Memcached. The current time and the cache time interval are compared, and if the sliding expiration parameter is not exceeded, it is re-deposited according to this parameter;

[TestMethod] public void Slidingwithresotre () {using (Memcachedclient client= New Memcachedclient ("enyim.com/memcached") ) {String key= Guid.NewGuid (). ToString ("N"); Object value=Guid.NewGuid (); {var slidingexpiration=timespan.fromseconds (15D); Object Wraper= new Slidingcachewraper<object>(value, slidingexpiration); Client.            Store (Storemode.set, Key, Wraper, Timespan.fromseconds (12D));            } thread.sleep (Timespan.fromseconds (10D));                {Object wraperobj; var exist=client.                Tryget (key, out wraperobj);                Assert.istrue (exist);                Assert.isnotnull (Wraperobj); Assert.istrue (Wraperobj is Slidingcachewraper<Object>); var wraper= (slidingcachewraper<object>) Wraperobj; Assert.isnotnull (Wraper.                Cache); Assert.AreEqual (value, Wraper.                Cache); Client. Store (Storemode.set, Key, Wraper, Wraper.            slidingexpiration);            } thread.sleep (Timespan.fromseconds (10D));                {Object wraperobj; var exist=client.                Tryget (key, out wraperobj);                Assert.istrue (exist);                Assert.isnotnull (Wraperobj); Assert.istrue (Wraperobj is Slidingcachewraper<Object>); var wraper= (slidingcachewraper<object>) Wraperobj; Assert.isnotnull (Wraper.                Cache); Assert.AreEqual (value, Wraper.                Cache); Client. Store (Storemode.set, Key, Wraper, Wraper.            slidingexpiration);            } thread.sleep (Timespan.fromseconds (20D));                {Object wraperobj; var exist=client.                Tryget (key, out wraperobj);                Assert.isfalse (exist);            Assert.isnull (Wraperobj); }        }    }

The store slides past the cache entry with an expiration parameter of 15 seconds, then two intervals of 10 second access and expects the cache not to expire, 20 seconds to access and expects the cache to expire, similar blocks of code are enclosed, and the test passes.

Part III Optimization

Admittedly the previous logic has achieved the effect of sliding expiration, but you should be aware of the problem, that is, for this requirement, each cache read, we have to re-write once, even if the Memcached performance is great, repeatedly hit the cache when the large number of writes also have IO and serialization pressure, which can be optimized.

Sliding expiration means that the cache entry will survive longer and cannot be solved by writing, my intuitive idea is to add a "pseudo expiration" to the cache entry, and the cache will not be updated when the cache time is increased from the start.

[TestMethod] public void Slidingwithresotre () {using (Memcachedclient client= New Memcachedclient ("enyim.com/memcached") ) {String key= Guid.NewGuid (). ToString ("N"); Object value=Guid.NewGuid (); Action Handler= () + ={Object wraperobj; var exist=client.                Tryget (key, out wraperobj); if(exist) {assert.isnotnull (wraperobj); Assert.istrue (Wraperobj is Slidingcachewraper<Object>); var wraper= (slidingcachewraper<object>) Wraperobj; Assert.isnotnull (Wraper.                    Cache); Assert.AreEqual (value, Wraper.                    Cache); TimeSpan Diffspan= DateTime.Now-Wraper.                    Cachingtime; Console.WriteLine ("Get Cache, diff {0:f0} sec., sliding {1:f0} sec.", Diffspan.totalseconds, Wraper.                    Slidingexpiration.totalseconds); //Current Time-set time > Sliding time, has expired                    if(Diffspan >Wraper. slidingexpiration) {Console.WriteLine ("Remove Cache"); Client.                        Remove (key);                    Return }                    //Current Time-set time > Swipe time/2, update cache                    if(Diffspan.add (Diffspan) >Wraper. slidingexpiration) {Console.WriteLine ("Restore Cahce"); Client. Store (Storemode.set, Key, Wraper, Wraper.                    slidingexpiration); }                }                Else{Console.WriteLine ("Overdue");            }            }; var slidingexpiration=timespan.fromseconds (15D); Object Extendedwraper= new Slidingcachewraper<object> (value, Timespan.fromseconds (Slidingexpiration.totalseconds *1.5)); Client.            Store (Storemode.set, Key, Extendedwraper, slidingexpiration); var random=new Random ();  for(inti =0; I <6; i++) {Thread.Sleep (Timespan.fromseconds ( the*Random.                Nextdouble ()));            Handler (); }        }    }

You can see that when the first half of the cache expiration time is accessed, it is not re-written, and is re-written when the second half of the cache expiration time is accessed, because the time set in Memcached is slidingexpiration * 1.5, and can still be exceeded after the expiration parameter is accessed to the cache. You should immediately determine that the cache has expired.

The test case uses a random time, that is, you may need to run the viewing effect multiple times, or you can manually set several Sleep time viewing effects, such as the following output:

Code is still very lengthy, type check, time difference calculation and so on, the package good interface see Jusfr.Caching.Memcached, the whole realization thought process can refer to the previous two articles, the code snippet is as follows:

public overrideBOOLTryget<t> (stringkey, out T entry)        {Object cacheentry; Boolean exist=Trygetobject (key, out cacheentry); if(!exist) {            //does not existEntry =default (T); Returnfalse; }        if(CacheEntry = =NULL) {            //exists but is nullEntry = (T) ((Object)NULL); Returntrue; }        if(CacheEntry is T) {//exist, return directlyEntry =(T) cacheentry; Returntrue; }        if(! (CacheEntry is expirationwraper<t>)) {            //The type is not T and is not expirationwraper<t> throw an exceptionthrow new InvalidOperationException (String.Format ("cache entry ' [{0}] ' type error, {1} or {2}?", Key, Cacheentry.gettype (). FullName, typeof (T).        FullName)); } var cachewraper= (expirationwraper<t>) CacheEntry; //indicates sliding expiration cache entry        if(Cachewraper.slidingexpiration = =cache.noslidingexpiration) {//absolute time expires, returnEntry =Cachewraper.value; Returntrue; } var Diffspan=DateTime.Now.Subtract (cachewraper.settingtime); //Current Time-set time > Sliding time, has expired        if(Diffspan >cachewraper.slidingexpiration) {Expire (key); Entry=default (T); Returnfalse; }        //Current Time-set time > Swipe time/2, update cache        if(Diffspan.add (Diffspan) >cachewraper.slidingexpiration) {entry=Cachewraper.value;        Overwrite (Key, Cachewraper.value, cachewraper.slidingexpiration); } entry=Cachewraper.value; Returntrue; }

This completes the optimization, if you have the good method, welcome the communication.

JUSFR original, reproduced please indicate from the blog park.

Memcached sliding expiration implementation and binary optimization