Based on Lucene3.5.0 how to get token from Tokenstream

By learning Lucene3.5.0 Doc document, the API modification of different release version number Lucene version number is analyzed. Finally, we found valuable modification information.

Lucene-2302:deprecated Termattribute and replaced by a new chartermattribute. The change was backwards compatible, so mixed new/old tokenstreams all work on the same char[] buffer independent of which Interface they use. Chartermattribute has shorter method names and implements Charsequence and appendable. This allows usage is like Java's StringBuilder in addition to direct char[] access. Also terms can directly be used in places where charsequence is allowed (e.g. regular expressions). (Uwe Schindler, Robert Muir)

The above information can be known that the original method has not been able to extract the token of the response

StringReader reader = new StringReader (s); Tokenstream ts =analyzer.tokenstream (s, reader); Termattribute ta = Ts.getattribute (termattribute.class);

By analyzing the API documentation information, Chartermattribute has become an interface for replacing Termattribute

So I wrote a sample to better extract tokens from the tokenstream.

Package Com.segment;import Java.io.stringreader;import Org.apache.lucene.analysis.analyzer;import Org.apache.lucene.analysis.token;import Org.apache.lucene.analysis.tokenstream;import Org.apache.lucene.analysis.tokenattributes.chartermattribute;import Org.apache.lucene.analysis.tokenattributes.termattribute;import Org.apache.lucene.util.attributeimpl;import Org.wltea.analyzer.lucene.ikanalyzer;public class Segment {public static string show (Analyzer A, String s) throws Excepti on {StringReader reader = new StringReader (s); Tokenstream ts = A.tokenstream (s, reader); String S1 = "", S2 = ""; Boolean hasnext= Ts.incrementtoken ();//token t = Ts.next (); while (Hasnext) {//attributeimpl TA = n EW Attributeimpl (); Chartermattribute ta = Ts.getattribute (chartermattribute.class);//termattribute ta = Ts.getattribute ( Termattribute.class); s2 = ta.tostring () + ""; S1 + = S2;hasnext = Ts.incrementtoken ();} return s1;} Public String Segment (string s) throws Exception {Analyzer a = new Ikanalyzer (); return Show (A, s);} public static void Main (string args[]) {string name = "I am a JJ, I love programming, my test example"; Segment s = new Segment (); String test = ""; try {System.out.println (test+s.segment (name));} catch (Exception e) {//TODO auto-generated catch Blocke.printstacktrace ();}}}

Based on Lucene3.5.0 how to get token from Tokenstream