Initial knowledge of Lucene full text Search

Lucene Overview
Lucene is an open source full-Text search engine toolkit, but it is not a full-text search engine, but rather a full-text search engine architecture that provides a complete query engine and indexing engine for some text analysis engines.

There are three types of data:

• Structured data (data with fixed format or finite length)
• Unstructured data
• Semi-structured data

SQL statement queries are generally used for structured data, while unstructured data has sequential and full-text retrieval.

Lucene File Structure
hierarchy: document, section---, index--word
The document is an atomic unit of Lucene indexing and searching, and the document is a container that includes one or more domains, and the domain really includes the content being searched, and the domain value is processed by the word segmentation technique to get multiple lexical elements.

Lucene Index Creation
Three steps to create an index: data to retrieve (Document), Word breaker (Analyzer), index creation (indexer)

//创建索引关键类//IndexWriteIndexWrite indexWrite=new IndexWrite(directory,indexWriteConfig);//DirectoryDirectory directory=FSDirectory.open(new File("C://index));//Analyzer  创建标准分词器Analyzer analyzer=new StandardAnalyzer(Version.LUCENE_43);//DocumentDocument doc=new Document();//Fileddoc.add(new TextField("filedname","测试",Store.YES));

Java Implementation Index creation

 PackageCom.lucene;ImportOrg.apache.lucene.analysis.Analyzer;ImportOrg.apache.lucene.analysis.standard.StandardAnalyzer;Importorg.apache.lucene.document.*;ImportOrg.apache.lucene.index.IndexWriter;ImportOrg.apache.lucene.index.IndexWriterConfig;ImportOrg.apache.lucene.store.Directory;ImportOrg.apache.lucene.store.FSDirectory;ImportOrg.apache.lucene.util.Version;ImportJava.io.File;ImportJava.io.IOException;/** * Created with IntelliJ idea. * User:yen * DATE:2016/8/17 * time:08:26 * */** * Index creation * / Public  class indexcreate {     Public Static void Main(string[] args)throwsIOException {//Specify Word segmentation technology, which uses standard participleAnalyzer analyzer=NewStandardAnalyzer (version.lucene_43);configuration information for//indexwriterIndexwriterconfig indexwriterconfig=NewIndexwriterconfig (Version.lucene_43,analyzer);//How to open an index: No, create it, open it .Indexwriterconfig.setopenmode (IndexWriterConfig.OpenMode.CREATE_OR_APPEND); Directory directory=NULL; IndexWriter indexwriter=NULL;Try{//Determine the location of the index file here is the local file storeDirectory= Fsdirectory.open (NewFile ("C://index"));//Unlock If index is locked            if(indexwriter.islocked (directory))            {indexwriter.unlock (directory); }//The Operation object for the specified index is indexwriteIndexWriter =NewIndexWriter (directory, indexwriterconfig); }Catch(Exception e)        {E.printstacktrace (); }finally{Indexwriter.close ();        Directory.close (); } Document doc1=NewDocument ();//stringfield DomainDoc1.add (NewStringfield ("id","ABC", Field.Store.YES));//textfield domain, using the specified word segmentation techniqueDoc1.add (NewTextField ("Content","Lucene test", Field.Store.YES));//intfield DomainDoc1.add (NewIntfield ("num",1, Field.Store.YES));//write the document to the indexIndexwriter.adddocument (Doc1);    Indexwriter.commit (); }}

Lucene Index Retrieval
Four steps for index retrieval: Search keywords (Keywords), Word breaker (Analyzer), index Search, return results.

Java Implements index retrieval

Packagecom. Lucene;import org. Apache. Lucene. Analysis. Analyzer;import org. Apache. Lucene. Analysis. Standard. StandardAnalyzer;import org. Apache. Lucene. Document. Document;import org. Apache. Lucene. Index. Directoryreader;import org. Apache. Lucene. Queryparser. Classic. ParseException;import org. Apache. Lucene. Queryparser. Classic. Queryparser;import org. Apache. Lucene. Search. Indexsearcher;import org. Apache. Lucene. Search. Query;import org. Apache. Lucene. Search. Topdocs;import org. Apache. Lucene. Store. Directory;import org. Apache. Lucene. Store. Fsdirectory;import org. Apache. Lucene. Util. Version;Import Java. IO. File;Import Java. IO. IOException;/** * Created with IntelliJ idea. * User:yen * DATE:2016/8/17 * time:08:42 * *public class Indexsearch {public static void main (string[] args) {Directory directory=null;try {//index hard disk storage path directory= fsdirectory. Open(New File ("C://index"));Read Index Directoryreader Directoryreader=directoryreader. Open(directory);Create index Retrieve object Indexsearcher search=new Indexsearcher (Directoryreader);Word Segmentation Technology Analyzer analyzer=new StandardAnalyzer (Version. LUCENE_43);Create a query Queryparser parser=new queryparser (Version. LUCENE_43,"Content", analyzer);Query Query=parser. Parse("Lucene case");Retrieves the index to get the pre-qualifiedTenRecords Topdocs Topdocs=search. Search(Query,Ten);if (Null!=topdocs) {System. out. println(Topdocs. Totalhits);for (int i =0; i < topDocs.scoreDocs.length; i++) {Document Doc=search. doc(Topdocs. ScoredocsI. doc);System. out. println("Id="+doc. Get("id"));System. out. println("Content="+doc. Get("Content"));}} Directory. Close();Directoryreader. Close();} catch (IOException e) {E. Printstacktrace();} catch (ParseException e) {E. Printstacktrace();}    }}

Lucene Word Breaker

Common word breakers:

• StandardAnalyzer Standard Word breaker
• Ikanalyzer third-party Chinese word segmentation technology based on Lucene
• Writespaceanalyzer Space Word Breaker
• Simpleanalyzer Simple word breaker
• Cjkanalyzer two-way word breaker
• Keywordanalyzer keyword word breaker
• Stopanalyzer is ignored word breaker

Packagecom. Lucene;import org. Apache. Lucene. Analysis. Analyzer;import org. Apache. Lucene. Analysis. Tokenstream;import org. Apache. Lucene. Analysis. CJK. Cjkanalyzer;import org. Apache. Lucene. Analysis. Core. Keywordanalyzer;import org. Apache. Lucene. Analysis. Core. Simpleanalyzer;import org. Apache. Lucene. Analysis. Core. Stopanalyzer;import org. Apache. Lucene. Analysis. Core. Whitespaceanalyzer;import org. Apache. Lucene. Analysis. Standard. StandardAnalyzer;import org. Apache. Lucene. Analysis. Tokenattributes. Chartermattribute;import org. Apache. Lucene. Util. Version;import org. Wltea. Analyzer. Lucene. Ikanalyzer;Import Java. IO. IOException;Import Java. IO. StringReader;/** * Created with IntelliJ idea. * User:yen * DATE:2016/8/17 * time:08:57 * *public class Analyzerdemo {private static String str="Lucene Case Development";public static void Main (string[] args) {//define a word breaker object Analyzer analyzer=null;Analyzer=new StandardAnalyzer (Version. LUCENE_43);Analyzerdemo. Show(analyzer);System. out. println("\ n .........................");Analyzerdemo. Show(New Ikanalyzer ());System. out. println("\ n .........................");Analyzerdemo. Show(New Whitespaceanalyzer (Version. LUCENE_43));System. out. println("\ n .........................");Analyzerdemo. Show(New Simpleanalyzer (Version. LUCENE_43));System. out. println("\ n .........................");Analyzerdemo. Show(New Cjkanalyzer (Version. LUCENE_43));System. out. println("\ n .........................");Analyzerdemo. Show(New Keywordanalyzer ());System. out. println("\ n .........................");Analyzerdemo. Show(New Stopanalyzer (Version. LUCENE_43));}//output participle result public static void Show (Analyzer analyzer) {StringReader stringreader=new StringReader (str);try {tokenstream Tokenstream=analyzer. Tokenstream("", StringReader);Tokenstream. Reset();Chartermattribute Termattribute=tokenstream. getattribute(Chartermattribute. Class);System. out. println("Word Segmentation technology:"+analyzer. GetClass());while (Tokenstream. Incrementtoken()) {System. out. Print(Termattribute. toString()+"|");}} catch (IOException e) {E. Printstacktrace();}    }}

Initial knowledge of Lucene full text Search