Lucene-based case development: real-time index management class Indexmanager

Reprint Please specify source: http://blog.csdn.net/xiaojimanman/article/details/44015983

Http://www.llwjy.com/blogd.php?id=5757ce8c007754704b563dd6a47ca1ca

Personal blog Station also build success, website: www.llwjy.com/blog.php, Welcome to spit Groove ~

In the previous blog, the real-time index of the implementation of the principle of a few simple introduction, here is introduced, how to use Lucene to achieve the management of the index (Lucene has achieved most of the functionality, we just need to encapsulate it again).

Break one by one

In Lucene4.3.1, the implementation of the real-time index, the need to indexwrite related operations delegated to Trackingindexwriter to handle, the specific code is implemented as follows:

PS: about how to create an index is not introduced here, you can refer to the previous blog or the full code behind the blog.

This.trackingindexwriter = new Trackingindexwriter (this.indexwriter);

Initialize the index management object at the same time, with the following code:

This.nrtmanager = new Nrtmanager (This.trackingindexwriter, New Searcherfactory ());

There are also two daemon threads to open: Memory index reread thread and memory data commit thread. Memory index Reread the frequency of the thread execution is the real-time index of the difference, because the memory of the data is not too much, so this delay is usually about more than 10 milliseconds; the memory data commit thread writes the in-memory data to disk without data loss. If you study the Lucene source of children's shoes may find that even if you do not perform a commit operation, to the memory of the data to a certain extent, will also write a portion of the data to disk, but restart the service this part of the data will be lost and will cause a series of problems, http:// bbs.csdn.net/topics/390677902 This address is the commit thread after the death of a series of problems caused by the interest of children's shoes can be understood.

Memory reread thread we only need to configure the following parameters to start, the code is as follows:

This.nrtmanagerreopenthread = new Nrtmanagerreopenthread (This.nrtmanager, Indexreopenmaxstalesec, INDEXREOPENMINSTALESEC); This.nrtManagerReopenThread.setName ("Nrtmanager Reopen Thread"); This.nrtManagerReopenThread.setPriority (Math.min (Thread.CurrentThread () getpriority () +2, thread.max_priority)); This.nrtManagerReopenThread.setDaemon (True); This.nrtManagerReopenThread.start ();

The memory data commit thread needs to write its own code implementation, then start the thread, the code is as follows:

Private class Indexcommitthread extends Thread{private boolean flag;public indexcommitthread (String name) {super (name);} @SuppressWarnings ("deprecation") public void Run () {flag = True;while (flag) {try {indexwriter.commit (); if (bprint) { System.out.println (New Date (). toLocaleString () + "\ T" + indexmanagername + "\tcommit");} TimeUnit.SECONDS.sleep (indexcommitseconds);} catch (IOException e) {e.printstacktrace ();} catch (Interruptedexception E1) {E1.printstacktrace ()}}}}

This.indexcommitthread = new Indexcommitthread (indexmanagername + "Index Commit Thread"); This.indexCommitThread.setDaemon (True); This.indexCommitThread.start ();

So how do you use indexsearcher like a normal index? Of course, the Nrtmanager class also provides the relevant methods to get the latest available indexsearcher, the code is as follows:

Public Indexsearcher Getindexsearcher () {try {return this.nrtManager.acquire ();} catch (IOException e) { E.printstacktrace (); return null;}}

Of course, don't forget to release after use, the code is as follows:

public void Release (Indexsearcher searcher) {try {nrtmanager.release (searcher);} catch (IOException e) {//TODO Auto-generated Catch block  e.printstacktrace ();}}

Alternative Singleton mode

In the previous blog, I also mentioned many times that loading an index is a fairly resource-intensive thing, so we can't load the index every time the index operation, so we have to use a singleton pattern to implement the Indexmanager class. The singleton pattern here is different from our common singleton pattern, the ordinary singleton mode has only one object, here the singleton pattern is that the class has more than one object, the following is a brief introduction of the alternative singleton mode.

By the end of the previous blog, you may notice that the configuration information about the index in the system is present in the HashSet object, which means that the Indexmanager class will instantiate how many times it depends on the HashSet object, that is, how many times the configuration file is instantiated for him to instantiate. In this case, how can you call a singleton mode? The singleton here is an index singleton, which means that an index has only one Indexmanager object, and there will be no two Indexmanager objects to manipulate the same index. The specific code is implemented as follows:

/** * Initialization on Demand holder initialization Indexmanager */private static class Lazyloadindexmanager {private static final Ha shmap<string, indexmanager> Indexmanager = new hashmap<string, indexmanager> (), Static {for (ConfigBean ConfigBean:IndexConfig.getConfigBean ()) {Indexmanager.put (Configbean.getindexname (), New Indexmanager (Configbean) );}}} /**   * @Description: Indexmanager Private Construction method * @Author: Lulei   * @Version: 1.1.0   */private Indexmanager ( Configbean Configbean) {//...} public static Indexmanager Getindexmanager (String indexname) {return LazyLoadIndexManager.indexManager.get (indexname );}

This allows us to get the Indexmanager object to the index by the index name.

Truth

Said so much, the following will be the source of the indexmanager attached to the last, interested in children's shoes can try (there are some other methods, I believe not to introduce also can see understand)

 /** * @Description: Index management class */package Com.lulei.lucene.index.manager; Import Java.io.file;import java.io.ioexception;import Java.util.date;import Java.util.hashmap;import Java.util.concurrent.timeunit;import Org.apache.lucene.analysis.analyzer;import Org.apache.lucene.index.indexwriter;import Org.apache.lucene.index.indexwriterconfig;import Org.apache.lucene.index.indexwriterconfig.openmode;import Org.apache.lucene.search.indexsearcher;import Org.apache.lucene.search.nrtmanager;import Org.apache.lucene.search.nrtmanager.trackingindexwriter;import Org.apache.lucene.search.nrtmanagerreopenthread;import Org.apache.lucene.search.searcherfactory;import Org.apache.lucene.store.directory;import Org.apache.lucene.store.niofsdirectory;import Org.apache.lucene.util.version;import Com.lulei.lucene.index.model.configbean;import Com.lulei.lucene.index.model.indexconfig;public class Indexmanager {private IndexWriter indexwriter;// Update the Indexwriterprivate trackingindexwriter of the index file Trackingindexwriter///index file using the word breaker private Analyzer analyzer;//index management Object Private Nrtmanager nrtmanager;//index reread thread private Nrtmanagerreopenthread nrtmanagerreopenthread;//index written to disk thread private Indexcommitthread indexcommitthread;//index address private String indexpath;// Index reread maximum, minimum interval private double indexreopenmaxstalesec;private double indexreopenminstalesec;//index commit time private int indexcommitseconds;//index name Private String Indexmanagername;//commit If relevant information is output private Boolean bprint = true;/** * Initialization on Demand holder initialization Indexmanager */private static class Lazyloadindexmanager {private static final HashMap <string, indexmanager> Indexmanager = new hashmap<string, indexmanager> (), Static {for (Configbean ConfigBean:IndexConfig.getConfigBean ()) {Indexmanager.put (Configbean.getindexname (), New Indexmanager (Configbean) );}}} /** * @Description: Indexmanager Private Construction method * @Author: Lulei * @Version: 1.1.0 */private indexmanager (Configbean configbean) {//Set related properties Analyzer = Configbean.getanalyzer (); Indexpath = Configbean.getindexpath (); indexmanagername = Configbean.getindexname (); indexreopenmaxstalesec = Configbean.getindexreopenmaxstalesec (); Indexreopenminstalesec = Configbean.getindexreopenminstalesec (); indexcommitseconds = Configbean.getindexcommitseconds (); bprint = Configbean.isbprint (); String indexfile = indexpath + indexmanagername + "/";//Create or open index indexwriterconfig indexwriterconfig = new Indexwriterconfi G (version.lucene_43, analyzer); Indexwriterconfig.setopenmode (openmode.create_or_append);D irectory Directory = null try {directory = Niofsdirectory.open (indexfile)), if (indexwriter.islocked (directory)) {Indexwriter.unlock ( directory);} This.indexwriter = new IndexWriter (directory, indexwriterconfig); this.trackingindexwriter = new Trackingindexwriter ( This.indexwriter); This.nrtmanager = new Nrtmanager (This.trackingindexwriter, New Searcherfactory ());} catch (IOException e) {e.printstacktrace ();} Open daemon This.setthread ();} /** * @Author: Lulei * @Description: CREATE index management thread */private void Setthread () {This.nrtmanagErreopenthread = new Nrtmanagerreopenthread (This.nrtmanager, indexreopenmaxstalesec, indexreopenminstalesec); This.nrtManagerReopenThread.setName ("Nrtmanager Reopen Thread"); This.nrtManagerReopenThread.setPriority (math.min (Thread.CurrentThread (). GetPriority () +2, thread.max_priority)); This.nrtManagerReopenThread.setDaemon (True); This.nrtManagerReopenThread.start (); this.indexcommitthread = new Indexcommitthread (Indexmanagername + "Index Commit Thread"); This.indexCommitThread.setDaemon (true); This.indexCommitThread.start ();} /** * @return * @Author: Lulei * @Description: Restart index commit thread */public String setcommitthread () {try {if (This.indexcommitt Hread.isalive ()) {return "Is Alive";} This.indexcommitthread = new Indexcommitthread (indexmanagername + "Index Commit Thread"); This.indexCommitThread.setDaemon (True); This.indexCommitThread.start ();} catch (Exception e) {e.printstacktrace (); return "Failed";} return "Reload";} /** * @Description: Index Commit thread * @Author: Lulei * @Version: 1.1.0 */private classIndexcommitthread extends Thread{private boolean flag;public indexcommitthread (String name) {super (name);} @SuppressWarnings ("deprecation") public void Run () {flag = True;while (flag) {try {indexwriter.commit (); if (bprint) { System.out.println (New Date (). toLocaleString () + "\ T" + indexmanagername + "\tcommit");} TimeUnit.SECONDS.sleep (indexcommitseconds);} catch (IOException e) {e.printstacktrace ();} catch (Interruptedexception E1) {E1.printstacktrace ()}}}} /** * @return Indexmanager * @Author: Lulei * @Description: Get index management class */public static Indexmanager Getindexmanager (String IndexName) {return LazyLoadIndexManager.indexManager.get (indexname);} /** * @ @Description: Release indexsearcher resource * @param searcher */public void release (Indexsearcher searcher) {try { Nrtmanager.release (searcher);} catch (IOException e) {//TODO auto-generated catch block E.printstacktrace ();}} /** * @return Indexsearcher * @Author: Lulei * @Description: Return Indexsearcher object, after use, call the release method to release */public Indexsearc Her getindexseArcher () {try {return this.nrtManager.acquire ();} catch (IOException e) {e.printstacktrace (); return null;}} Public Nrtmanager Getnrtmanager () {return this.nrtmanager;} Public IndexWriter Getindexwriter () {return this.indexwriter;} Public Trackingindexwriter Gettrackingindexwriter () {return this.trackingindexwriter;} Public Analyzer Getanalyzer () {return Analyzer;} /** * @return * @Author: Lulei * @Description: Gets the number of record bars in the index */public int getindexnum () {return Indexwriter.numdocs ();}}

PS: Recently found other sites may be reproduced on the blog, there is no source link, if you want to see more about Lucene-based case development please click here. Or visit the URL http://blog.csdn.net/xiaojimanman/article/category/2841877 or http://www.llwjy.com/blog.php

Lucene-based case development: real-time index management class Indexmanager