Lucene Custom Filters

First introduce the query and filter the difference and contact, in fact, query (various queries) and filtering (various filter) very similar between, can say as long as the use of query can be done, with the filter can also be done, they can be converted to each other, The biggest difference is that the result set returned with the filter does not have a scoring action, and the results returned with query are correlated scores, so when we have a business that has nothing to do with the scoring operation, it takes precedence over the filter operation to get better performance. In fact, this is the difference between the Q parameter and the FQ parameter in SOLR.


below, start to get to the point, before this, the scattered fairy still like the old cliché first to understand that Lucene has a general knowledge of the filter



now, let's look at the specifics of how this is implemented in the code, first look at our test data

Java code

2. ID score bookname ename type price date
4. 1 1 Ethereal Journey Pmzl novel 52.23 201005
6. 2 1 Kingdoms sgyy novel 36.13 201207
8. 3 1 Database Combat Sjksz technology 77.13 200811
10. 4 1 Series Bible BCBD technology 100.3 200501
12. 5 1 Workplace Relations ZCGXL career 36.59 200501
14. 6 1 Healthy Living jksh life 20.47 200008
16. 7 1 See the Essence KQBZ Society 10.37 201004
18. 8 1 programming, Programming BCBC Society 10.37 201004

Core Code

Java code

1. Use filter When the last true contains the boundary part, false when the boundary part is not included
2. When the second-to-last is true, contains the query boundary, false when it does not contain
3. Termrangefilter filter=new termrangefilter ("ename", New Bytesref ("H"), New Bytesref ("n"), True, true);
4. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);//default no Sort mode

Output Results

Java code

1. 6 1 Healthy Living jksh life 20.47 200008
2. 7 1 See the Essence KQBZ Society 10.37 201004

Core Code

Java code

1. Numericrangefilter<double> Filter=numericrangefilter.newdoublerange ("Price", 10D, 40D, True, false);
2. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);//default no Sort mode

Output Results

Java code

2. 2 1 Kingdoms sgyy novel 36.13 201207
4. 5 1 Workplace Relations ZCGXL career 36.59 200501
6. 6 1 Healthy Living jksh life 20.47 200008
8. 7 1 See the Essence KQBZ Society 10.37 201004
10. 8 1 programming, Programming BCBC Society 10.37 201004

Core Code

Java code

1. Using Cache filtering
2. Filter Filter=fieldcacherangefilter.newdoublerange ("Price", 20D, 50D, True, true);
3. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);//default no Sort mode

Output Results

Java code

1. 2       1        Kingdoms          sgyy         Fiction        36.13       201207         
2. 5       1        Workplace Relations        zcgxl        Workplace        36.59       200501         
3. 6       1         Healthy Living         jksh         Life       20.47       200008     

Core Code

Java code

1. Cache domain filter for specific categories
2. Filter filter=new fieldcachetermsfilter ("type", New string[]{"Technology", "Society"});
3. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);//default no Sort mode

Output Results

Java code

2. 3 1 Database Combat Sjksz technology 77.13 200811
4. 4 1 Series Bible BCBD technology 100.3 200501
6. 7 1 See the Essence KQBZ Society 10.37 201004
8. 8 1 programming, Programming BCBC Society 10.37 201004

Core Code

Java code

1. Use the Querywrapperfilter class to wrap a query
2. Querywrapperfilter filter=new Querywrapperfilter (New Termquery ("type", "technology"));
3. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);//default no Sort mode

Output Results

Java code

1. 3 1 Database Combat Sjksz technology 77.13 200811
2. 4 1 Series Bible BCBD technology 100.3 200501

Finally, I look at how to inherit the filter base class, to customize our own filter, the custom filter, although at some point, the function is very powerful and flexible, but there are a few shortcomings, we know 1, the guarantee is the content of non-repeating fields, such as the primary key, if repeated, Default returns the first as a result set showing 2, which guarantees that the content cannot be participle, and if the field is a word breaker, some incorrect results may occur.
Custom Filter Class

Java code

2. Package com.sanjiesanxian.test;
6. Import java.io.IOException;
8. Import Java.util.BitSet;
12. Import Org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
14. Import Org.apache.lucene.index.AtomicReaderContext;
16. Import Org.apache.lucene.index.DocsEnum;
18. Import Org.apache.lucene.index.Term;
20. Import Org.apache.lucene.search.DocIdSet;
22. Import Org.apache.lucene.search.Filter;
24. Import Org.apache.lucene.util.AttributeSource;
26. Import org.apache.lucene.util.Bits;
28. Import Org.apache.lucene.util.DocIdBitSet;
30. Import Org.apache.lucene.util.FixedBitSet;
32. Import Org.apache.lucene.util.OpenBitSet;
36. /***
38. *^_^  ^_^  ^_^
40. * Custom Filters
42. * @author
44. * */
46. public class Mycustomfilter extends filter{
50. Public Mycustomfilter () {
52. TODO auto-generated Constructor stub
54. }
58. Private string[] terms;//limit the returned data dictionary
60. Public Mycustomfilter (String ... terms) {
62. TODO auto-generated Constructor stub
64. this.terms=terms;
66. }
68. @Override
70. Public Docidset Getdocidset (Atomicreadercontext arg0, Bits arg1)
72. Throws IOException {
74. Fixedbitset bits=new Fixedbitset (Arg0.reader (). Maxdoc ());//Get not all docid including the non-deleted
76. The relative cardinality of the int base=arg0.docbase;//segment, guaranteeing a correct position relative to multiple segments
78. int Limit=base+arg0.reader (). Maxdoc ();//Calculate Maximum limit value
80. for (String s:terms) {
82. Docsenum Doc=arg0.reader (). Termdocsenum (New term ("id", s));//must be unique and not repeatable
84. Guarantee is a single non-repeating term, if repeated, the default will take the first as the return result set, the term after the word also does not use the custom term
86. if (Doc.nextdoc ()!=-1) {
88. Bits.set (Doc.docid ());//To add a DocID loop that matches a conditional constraint to bits
90. }
92. }
94. return bits;
96. }
114. }

test the query code

Java code

1. Mycustomfilter filter=new Mycustomfilter ("3", "5", "2");//Optionally specify more than 1 items that need to be filtered
2. Topdocs Topdocs=searcher.search (New Matchalldocsquery (), filter,10000);

Output Results

Java code

1. 2       1        Kingdoms          sgyy         Fiction        36.13       201207         
2. 3       1        Database Combat        sjksz        Technology        77.13       200811         
3. 5       1         Workplace Relations        zcgxl        Workplace        36.59       200501    

Although the custom filter has drawbacks, it can play a very flexible role in some scenarios, especially for fields that do not have a word breaker.

Lucene Custom Filters