Distributed Cache Series Guava Cache

Guava is an open source Java framework for Google, and its GitHub address is Https://github.com/google/guava. The Guava project contains several core libraries that are widely relied on by Google's Java projects, such as: collections [Collections], cache [caching], native type support [primitives supports], concurrent libraries [concurrency Libraries], common annotations [common annotations], string handling [string processing], I/O, and so on. All of these tools are used by Google's engineers every day in product services. One of the caching is one of my favorite modules, and share some of my insights on guava cache today.

Guava Cache Usage Profile

      The guava cache uses the Cachebuilder class to construct two different cache load modes cacheloader,callable with the builder pattern, which is based on the value of key. The difference is that Cacheloader is broadly defined for the entire cache and can be considered a uniform method of load value based on the key value, while the callable is more flexible, allowing you to specify the load method at get. Look at the following code

cache<string,object> cache = Cachebuilder.newbuilder ()                . Expireafterwrite (timeunit.seconds). MaximumSize (+). Build ();         Cache.get ("Key", new Callable<object> () {//callable loaded            @Override public            Object call () throws Exception { C4/>return "value";            }        });        loadingcache<string, object> Loadingcache = Cachebuilder.newbuilder ()                . Expireafteraccess (30, timeunit.seconds). MaximumSize (5)                . Build (New cacheloader<string, object> () {                    @Override                    public Object load (String key) throws Exception {                        return "value";                    }                });

There are several parameters in this expireafterwrite, expireafteraccess, maximumsize in fact, these are defined as expiration policy. Expireafterwrite for a period of time the cache may change the scene first. Expireafteraccess is an access operation that includes Expireafterwrite, because both the read and write operations are defined. In addition , expireafteraccess,expireafteraccess are subject to maximumsize restrictions. When the number of caches exceeds maximumsize, the guava cache will eliminate data that was not recently written or accessed by the LRU algorithm. maximumsize Here means that the number of caches is not the size of the cache occupying memory. You can configure the Maximumweight parameter if you want to limit the size of the cache to occupy memory .

Look at the code:

  Cachebuilder.newbuilder (). Weigher (New weigher<string, object> () {              @Override public              int weigh (String Key, Object value) {                  return 0;  The Value.size ()              }          ). Expireafterwrite (Timeunit.seconds). Maximumweight (+). Build ();

Weigher returns each cache value that occupies the size of the memory, which is defined by the consumer itself, and is determined to be no longer changed after it is put into memory. Maximumweight defines the upper limit of the sum of all the weigher that are added by the cache value.

Note that Maximumweight and maximumsize can only take effect one is not used at the same time!

Design of the guava cache

Guava Cache as a widely used caching component, what is the design of its extraordinary?

First look at the cache class implementation definition

Class localcache<k, v> extends Abstractmap<k, v> implements Concurrentmap<k, v> {...}

We saw the concurrentmap, so we know a little bit of guava cache based on Concurrenthashmap design. So the advantage of Concurrenthashmap it also has. Since the realization of the concurrentmap then look at the guava cache segment implementation is what?

We see the segment nature of the guava cache as a reentrantlock. The Table,wirtequeue,accessqueue definition attribute is defined internally. Where table is an array of referenceentry atomic classes that store the contents of the cache. The Wirtequeue store is a write record of table, Accessqueue is the access record. The guava cache's expireafterwrite,expireafteraccess is implemented with this two queue.

Knowing the approximate storage structure of the guava cache, here's a more in-depth look at the operation of the cache.

Put (Key,val) operation.

  Public V put (K key, V value) {    checknotnull (key);    Checknotnull (value);    int hash = hash (key);    return Segmentfor (hash). Put (key, hash, value, false);  }

Set the approximate process of caching: according to the key hash to the corresponding segment, then to segment locking lock (), and then get segment.table corresponding node

int index = hash & (Table.length ()-1); Referenceentry<k, v> first = Table.get (index);

Then the queue process is similar to the HASHMAP process. After the team will also carry out related operations such as update accessqueue and wiritequeue, accumulate totalweight

void Recordwrite (referenceentry<k, v> entry, int weight, long now) {      //We is already under lock, so drain the Recency queue immediately      drainrecencyqueue ();      Totalweight + = weight;      if (map.recordsaccess ()) {        entry.setaccesstime (now);      }      if (Map.recordswrite ()) {        entry.setwritetime (now);      }      Accessqueue.add (entry);      Writequeue.add (entry);    }

Get (key) operation.

The first step is to locate the segment first.

V Get (K key, cacheloader< Super K, V> loader) throws executionexception {    int hash = hash (Checknotnull (key)); 
   return segmentfor (hash). Get (key, hash, loader);  }

Determine the referenceentry existence of the key corresponding

  Referenceentry<k, v> e = getentry (key, hash);          if (E! = null) {            Long now = Map.ticker.read ();            V value = Getlivevalue (E, now);            if (value! = null) {              Recordread (e, now);              Statscounter.recordhits (1);              Return Schedulerefresh (E, key, hash, value, now, loader);            }            Valuereference<k, v> valuereference = E.getvaluereference ();            if (valuereference.isloading ()) {              return Waitforloadingvalue (E, Key, valuereference);            }          }

Getlivevalue (E, now) if NULL is returned to indicate that the current cache has expired, is not NULL when Recordread (E, now) records the latest access time to now, and then counts the hit ratio. Schedulerefresh (E, key, hash, value, now, loader) is equivalent to a double check, again checking that the cache has expired or there are no other threads in the update. If none of the old returns take the original value back, there is a call to the loader method to get the latest value and then return.
Note that if it is Loadingcache, and valuereference.isloading () is true, it indicates that another thread is updating the cache, and all other threads wait until this thread loading finished
To return.

   Key corresponding to the referenceentry does not exist: the cache is not loaded or has been removed.

      Return Lockedgetorload (key, hash, loader);

　The lockedgetorload execution logic is to first lock lock (), to determine if there are currently other threads in loading the cache, if there is waiting for it to load and then return. Do not perform loader yourself set the value into the cache and return.    

try {          //synchronizes on the entry to allow failing fast when a recursive load is          //detected. This could be circumvented if an entry was copied, but would fail fast most          //of the time.          Synchronized (e) {            return Loadsync (key, hash, loadingvaluereference, loader);          }        } finally {          Statscounter.recordmisses (1);        }

　　

Guava Cache's elimination strategy

Guava cache generally has four elimination strategies.

1, size-based Basic to the use of quantity.

When the number of caches exceedsCacheBuilder.maximumSize(long)设置的值时，优先淘汰最近没有使用或者不常用的元素。同理CacheBuilder.maximumWeight(long)也是一样逻辑。


2, Timed eviction based on time expulsion.

The elimination expireAfterAccess(long, TimeUnit) logic is similar to size-based only after you specify that the last read/update operation is over a specified duration.优先淘汰最近没有使用或者不常用的元素

expireAfterWrite(long, TimeUnit) 仅在指定上一次写/更新操作过了指定持续时间之后才考虑淘汰，淘汰逻辑与size-based是类似的。优先淘汰最近没有使用或者不常用的元素

3, reference-based eviction Basic to cite expulsion

after JDK1.2, Java extends the concept of references into strong references (strong Reference), soft references (Soft Reference), weak references (Weak refernce), virtual references (Phantom refere NCE). The four reference intensities are reduced in turn. These four references, in addition to strong references (strong Reference), can be guaranteed to be recycled when the corresponding object of the reference is used by the JVM for GC. So by using weak-referenced keys, or weak-referenced values, or soft-referenced values, the guava cache can be set to allow garbage collection:

• Cachebuilder.weakkeys (): Use weak reference store key. When a key has no other (strong or soft) reference, the cache entry can be garbage collected. Because garbage collection relies only on identities (= =), the cache with weak reference keys is used = = instead of the Equals comparison key.
• cachebuilder.weakvalues (): Stores values using weak references. When there are no other (strong or soft) references to the value, the cache entry can be garbage collected. Because garbage collection relies only on identities (= =), caches with weak reference values use = = instead of equals to compare values.
• cachebuilder.softvalues (): Stores values using soft references. Soft references are recycled only in response to memory needs, in the order of least recently used globally. Given the performance impact of using soft references, we generally recommend a more performance-predictive cache size qualification (using a soft-reference cache also uses = = instead of equals comparison values)

The benefit is that when memory resources are strained, memory can be freed up to cache. Attention! Cachebuilder If there is no indication that the default is a strong reference, the GC cannot be reclaimed if no element reaches the specified expiration time.

4. Display Delete

At any time, you can explicitly clear the cache entry instead of waiting for it to be recycled:

• Individual purge:cache.invalidate (key)
• Batch purge:cache.invalidateall (keys)
• Clear all Cache entries:cache.invalidateall ()

Mention how the guava cache triggers elements to be recycled. Guava element recycling is not the same as some of its frameworks, such as Redis,redis is an extra thread to reclaim elements. And guava is the way to carry out the get,put operation when the element is recycled. This can reduce overhead compared to the normal caching of thread-monitoring cleanup, but if the method is not called for a long time, it will cause the problem of freeing the memory space from being cleaned up in a timely manner. The main processing of four queue:1 during recycling. Keyreferencequeue;2. Valuereferencequeue;3. Writequeue;4. Accessqueue. The first two queues are added when WeakReference, SoftReference are garbage collected, and only need to traverse the entire queue to remove the corresponding items from the LocalCache. Here Keyreferencequeue store Referenceentry, and Valuereferencequeue store is valuereference. For the next two queues, you only need to check that the corresponding expire time is configured, and then find the entry that have been expire from scratch and remove them.

Overall, guava cache based on Concurrenthashmap's excellent design for reference, in high concurrency scenarios to support thread safety, using the Reference Reference command, to ensure that the GC can be recycled to the corresponding data, effectively save space At the same time, the write chain and access chain design, can be more flexible and efficient implementation of various types of cache cleanup strategy, including capacity-based cleanup, time-based cleanup, reference-based cleanup and so on;

Distributed Cache Series Guava Cache