How to obtain tokens from tokenstream Based on ipve3.5.0

Learn about how to modify the API of Lucene versions of different release versions by learning the doc documentation of javase3.5.0. Finally, we found valuable information for modification.

LUCENE-2302: deprecated termattribute and replaced by a new chartermattribute. the change is backwards compatible, so mixed new/old tokenstreams all work on the same char [] buffer independent of which interface they use. chartermattribute has shorter method names and implements charsequence and appendable. this allows usage like Java's stringbuilder in addition to direct char [] access. also terms can directly be used in places where charsequence is allowed (e.g. regular Expressions ). (UWE Schindler, Robert Smith)

The preceding information indicates that the response token cannot be extracted from the original method.

StringReader reader = new StringReader(s);TokenStream ts =analyzer.tokenStream(s, reader);TermAttribute ta = ts.getAttribute(TermAttribute.class);

According to the analysis of API documentation information, chartermattribute has become an interface for replacing termattribute.

Therefore, I have compiled an example to better extract tokens from tokenstream.

Package COM. segment; import Java. io. stringreader; import Org. apache. lucene. analysis. analyzer; import Org. apache. lucene. analysis. token; import Org. apache. lucene. analysis. tokenstream; import Org. apache. lucene. analysis. tokenattributes. chartermattribute; import Org. apache. lucene. analysis. tokenattributes. termattribute; import Org. apache. lucene. util. attributeimpl; import Org. wltea. analyzer. lucene. ikanalyzer; public class segment {public static string show (analyzer A, string s) throws exception {stringreader reader = new stringreader (s); tokenstream Ts =. tokenstream (S, Reader); string S1 = "", S2 = ""; Boolean hasnext = ts. incrementtoken (); // token T = ts. next (); While (hasnext) {// attributeimpl TA = new attributeimpl (); chartermattribute TA = ts. getattribute (chartermattribute. class); // termattribute TA = ts. getattribute (termattribute. class); S2 = TA. tostring () + ""; S1 + = S2; hasnext = ts. incrementtoken ();} return S1;} Public String segment (string s) throws exception {analyzer A = new ikanalyzer (); return show (A, S );} public static void main (string ARGs []) {string name = "I am Jun Jie, I love programming, my samples trial example"; Segment S = new segment (); string test = ""; try {system. out. println (test + S. segment (name);} catch (exception e) {// todo auto-generated catch blocke. printstacktrace ();}}}