Ehcache BigMemory: Getting rid of GC puzzles (RPM)

Problem

Using Java Open source projects often requires tuning the JVM to optimize the GC. For GC, if the object is a short-term object, then the JVM is relatively easy to optimize, if it comes to a project like SOLR using its own Java cache, then the GC is severely limited to the cache because the cache object is not a short-term object, so that young GC is often accompanied by a large number of memory object copies, Severely affects GC performance.

Ehcache BigMemory

Java's memory management mechanism is extremely inappropriate for the cache, and the best approach is to use the JNI implementation of the cache system. Another common approach: Ehcache BigMemory (http://ehcache.org/). BigMemory extends Ehcache ' s ' capabilities with a off-heap store that frees-from GC ' s constraints.

For bigmemory, download the free 32G restricted version directly (note: Each JVM process uses up to 32G of off-heap space, which is sufficient for most applications).

For information on how to use, see official documentation: http://terracotta.org/documentation/4.0/bigmemorygo/get-started

Use the example to refer to the sample that comes with your code: Bigmemory-go-4.0.0/code-samples/src/main/java/com/bigmemory/samples

The sample code is missing the compilation configuration Build.xml, and the following build.xml is placed in bigmemory-go-4.0.0/code-samples to compile the sample code using ant:

<project name= "Bigmemory" basedir= "." > <property name= "build.dir" value= "${basedir}/build"/> <property name= "Src.class.dir" value= "$ {build.dir} "/> <property name=" Src.dir "value=" ${basedir}/src "/> <property name=" Lib.dir "Valu E= "${basedir}/.                /lib "/> <property name=" Config.dir "value=" ${basedir}/config "/> <path id=" Base.classpath "> <pathelement location= "${src.class.dir}"/> <pathelement location= "${config.dir}"/                > <fileset dir= "${lib.dir}" > <include name= "**/*.jar"/>  </fileset> </path> <path id= "Classpath" > <path refid= "Base.classpath"                /> <fileset dir= "${lib.dir}" > <include name= "**/*.jar"/>        </fileset> </path> <path id= "Build.src.path" >        <pathelement location= "${src.class.dir}"/> </path> <target name= "clean" description= "Clean" > <delete dir= "${build.dir}"/> </target> <target name= "Compile" dep ends= "clean" description= "compile" > <mkdir dir= "${src.class.dir}"/> <javac srcd                        Ir= "${src.dir}" destdir= "${src.class.dir}" source= "1.6" debug= "on" encoding= "Utf-8" Includeantruntime= "false" > <classpath refid= "Base.classpath"/> </javac> </target> <                         Target Name= "jar" depends= "compile" description= "jar" > <jar destfile= "${build.dir}/bigmemory.jar" > <fileset dir= "${src.class.dir}" > <exclude name= "**/timer/* * "/> </fileset> </jar> </target></project>

Configuration Description: Bigmemory-go-4.0.0/config-samples/ehcache.xml details the configuration parameters.

Limit:

1, storage objects all use java.io.Serializable to do serialization and deserialization, performance loss.

2, Off-heap space once allocated cannot be adjusted.

SOLR Cache

The Ehcache bigmemory was introduced to optimize the SOLR cache. The following code is the Ehcache cache class implemented based on the SOLR cache base class, using the same as in SOLR. Fastlrucache, ehcache external configuration file is required.

Package Org.apache.solr.search;import Org.apache.solr.common.solrexception;import Org.apache.solr.common.util.namedlist;import Org.apache.solr.common.util.simpleorderedmap;import Org.apache.solr.core.solrcore;import Java.util.*;import Java.util.concurrent.atomic.atomiclong;import Java.io.ioexception;import Java.net.url;import Net.sf.ehcache.cache;import Net.sf.ehcache.cachemanager;import Net.sf.ehcache.element;import Net.sf.ehcache.config.cacheconfiguration;import Net.sf.ehcache.config.configuration;import net.sf.ehcache.config.memoryunit;/** * @version $Id: EhCacheWrapper.java 2013-03-27 zhenjing Chen $ */public class Ehcachewrapper implements Solrcache {/* An instance of this class would be Sha  Red across multiple instances * of an LRUCache at the same time.   Make sure everything is thread safe.    */private static class Cumulativestats {Atomiclong lookups = new Atomiclong ();    Atomiclong hits = new Atomiclong ();    Atomiclong inserts = new Atomiclong (); Atomiclong EvictiONS = new Atomiclong ();  } private cumulativestats stats;  per instance stats. The synchronization used for the map would also be//used for updating these statistics (and hence they is not Atomiclon  GS private long lookups;  Private long hits;  Private long inserts;  private long evictions;  Private long warmuptime = 0;  Private CacheManager manager = null;  Private Cache map;  private String name;  Private String Cache_name;  private int autowarmcount;  Private state State;  Private Cacheregenerator regenerator;  Private String description= "Eh LRU Cache";  private static int cache_index = 0;  private static map<string, cachemanager> managerpool = null;  private static map<string, integer> managerflag = null;  private static CacheManager managertemplate = null;    static{Managerpool = new hashmap<string, cachemanager> ();    Managerflag = new hashmap<string, integer> ();  Managertemplate = new CacheManager ("/data/conf/ehcache.xml"); } Private Cache GetCache () {//Use cache pool set<string> Set = Managerflag.keyset ();    Iterator<string> it = Set.iterator ();        while (It.hasnext ()) {String cachename = It.next ();                if (Managerflag.get (cachename) = = 0) {//not used manager = Managerpool.get (cachename);                        System.out.println ("Ehcachewrapper Cache Name (Pool):" + cachename);            Managerflag.put (CacheName, 1);            Cache_name = CacheName;        Return Manager.getcache (CacheName);    }}//Add zhenjing String cachename = name + Cache_index;         System.out.println ("Ehcachewrapper Cache Name:" + cachename);    Create cache from template Cache orig = Managertemplate.getcache (name);    Cacheconfiguration configtmp = Orig.getcacheconfiguration ();    Configtmp.setname (CacheName);    Configuration managerconfiguration = new configuration ();    Managerconfiguration.setname (CacheName); Manager = new CacheManager (ManagerconfigurAtion.cache (configtmp));    Put to Cache pool Managerflag.put (cachename, 1);    Managerpool.put (CacheName, manager);    Get Cache cache_index++;    Cache_name = CacheName;  Return Manager.getcache (CacheName);    } public Object init (Map args, Object persistence, Cacheregenerator regenerator) {state=state.created;    This.regenerator = regenerator;    Name = (String) args.get ("name");    String str = (string) args.get ("size"); Final int limit = Str==null?    1024:integer.parseint (str);    str = (String) args.get ("InitialSize");    Final int initialsize = Math.min (str==null 1024:integer.parseint (str), limit);    str = (String) args.get ("Autowarmcount"); Autowarmcount = Str==null?    0:integer.parseint (str);    Get Cache map = GetCache ();    cacheconfiguration config = map.getcacheconfiguration (); Description = "Eh LRU Cache (maxbyteslocaloffheap=" + config.getmaxbyteslocaloffheap () + ", maxbyteslocalheap=" + config.g Etmaxbyteslocalheap () + ", Maxentrieslocalheap= "+ config.getmaxentrieslocalheap () +") "; if (persistence==null) {//Must be the first time a cache of this type is being created persistence = new Cumul    Ativestats ();    } stats = (cumulativestats) persistence;  return persistence;  } public String name () {return name;    } public int size () {synchronized (map) {return map.getsize (); }} public object put (object key, object value) {synchronized (map) {if (state = = state.live) {stats.i      Nserts.incrementandget ();      }//Increment local inserts regardless of state???      It does make it more consistent with the current size ... inserts++;      Map.put (New Element (Key,value));  return null;    Fake the previous value associated with key.      }} public object get (object key) {synchronized (map) {Element val = map.get (key);        if (state = = state.live) {//Only increment lookups and hits if we are LIVE.        lookups++; Stats.lookUps.incrementandget ();          if (val!=null) {hits++;          Stats.hits.incrementAndGet (); SYSTEM.OUT.PRINTLN (name + "EH Cache hit.        key= "+ key.tostring ());      }} if (val = = null) return null;    return Val.getobjectvalue ();    }} public void Clear () {synchronized (map) {map.removeall ();  }} public void SetState (state state) {this.state = state;  } public State getState () {return state;  } public void warm (Solrindexsearcher searcher, Solrcache old) throws IOException {return;    } public void Close () {clear ();    Flag un-used managerflag.put (cache_name, 0);  System.out.println ("Ehcachewrapper Cache Name (Reuse):" + cache_name); }////////////////////////Solrinfombeans methods//////////////////////public String GetName () {return ehcachewrap  Per.class.getName ();  } public String GetVersion () {return solrcore.version;  } public String GetDescription () {return description; } public CatEgory GetCategory () {return category.cache;  Public String Getsourceid () {return ' NULL ';  Public String GetSource () {return ' NULL ';  } public url[] Getdocs () {return null;  }//Returns a ratio, not a percent.    private static String Calchitratio (long lookups, long hits) {if (lookups==0) return "0.00";    if (lookups==hits) return "1.00";   int hundredths = (int) (hits*100/lookups);    Rounded down if (hundredths <) return "0.0" + hundredths;    Return "0." + hundredths;    /*** code to produce a percent, if we want it ... int ones = (int) (hits*100/lookups);    int tenths = (int) (hits*1000/lookups)-ones*10;    return integer.tostring (ones) + '. ' + tenths;    /} public Namedlist Getstatistics () {namedlist lst = new Simpleorderedmap ();      Synchronized (map) {Lst.add ("lookups", lookups);      Lst.add ("hits", hits);      Lst.add ("Hitratio", Calchitratio (lookups,hits));      Lst.add ("inserts", inserts); Lst.add ("EvictionS ", evictions);    Lst.add ("Size", map.getsize ());    } lst.add ("Warmuptime", warmuptime);    Long clookups = Stats.lookups.get ();    Long chits = Stats.hits.get ();    Lst.add ("Cumulative_lookups", clookups);    Lst.add ("Cumulative_hits", chits);    Lst.add ("Cumulative_hitratio", Calchitratio (clookups,chits));    Lst.add ("Cumulative_inserts", Stats.inserts.get ());    Lst.add ("Cumulative_evictions", Stats.evictions.get ());  return LST;  } public String ToString () {return name + getstatistics (). ToString (); }}

External Ehcache.xml configuration:

<ehcache xmlns:xsi= "http://www.w3.org/2001/XMLSchema-instance" xsi:nonamespaceschemalocation= "http// Ehcache.org/ehcache.xsd "updatecheck=" false "monitoring=" AutoDetect "dynamicconfig=" true "name=" config "    ; <!--<cache name= "Filtercache" maxentrieslocalheap= "1024x768" eternal= "true" Overflowtooffhea P= "true" maxbyteslocaloffheap= "1g" > </cache> <cache name= "Fieldvaluecache" Maxentriesloc Alheap= "1024x768" eternal= "true" overflowtooffheap= "true" maxbyteslocaloffheap= "1g" > </cache&    Gt --<cache name= "Queryresultcache" maxentrieslocalheap= "1" eternal= "true" overflowtooffhe  Ap= "true" maxbyteslocaloffheap= "800m" > </cache> <!--ehcache not support Documentcache, encoding    Format error.     <cache name= "Documentcache" maxentrieslocalheap= "1024x768" eternal= "true" overflowtooffheap= "true"    maxbyteslocaloffheap= "1g" > </cache>--></ehcache> 

Http://www.cnblogs.com/zhenjing/p/Ehcache_BigMemory.html

Ehcache BigMemory: Getting rid of GC puzzles (RPM)