[Elasticsearch] Proximity match (ii)-multivalued field, proximity and correlation

Multi-Value field (multivalue fields)

Using phrase matching on multi-valued fields can produce erratic behavior:

put/my_index/groups/1  {"  names "  : [ " John Abraham ,  " Lincoln smith"  ]} 

Run a phrase query for Abraham Lincoln:

get/my_index/groups/_search{"  query : {"  match_phrase " : {  " names " : "  Abraham Lincoln }} "

Surprisingly, the document above matches the query. Even if Abraham and Lincoln are among the two names in the name array. This behavior occurs because of how the array is indexed in ES.

When John Abraham is parsed, it produces the following information:

• Location 1:john
• Location 2:abraham

Then when Lincoln Smith is parsed, it produces:

• Location 3:lincoln
• Location 4:smith

In other words, ES produces the same result as the list of entries generated by the above array analysis and the parsing of a single string, John Abraham Lincoln Smith. In our query, we query the adjacency Abraham and Lincoln, and the two entries do exist and are contiguous in the index, so the query matches.

Fortunately, there is an easy way to avoid this situation, through the Position_offset_gap parameter, which is configured in the field map:

delete/my_index/groups/put/my_index/_mapping/groups { "properties" : { "names" : { "type" : "string" , "position_offset_gap" : -}    }}

The Position_offset_gap setting tells ES to set a deviation value for each new element in the array. Therefore, when we index the name array above, we will produce the following result:

• Location 1:john
• Location 2:abraham
• Location 3:lincoln
• Location 4:smith

Now our phrase match will not match the document, because the distance between Abraham and Lincoln is 100. You must add a value of slop value of 100 to match.

The closer the better (Closer is better)

The phrase query (Phrase query) simply excludes documents that do not contain exact query phrases, while the proximity query (Proximity query)-a phrase query with a slop value greater than 0-will also take the proximity of the query entry into the final correlation _score. By setting a high slop value like 50 or 100, you can exclude documents that are too far away, but also give a higher score to the documents that are adjacent to those words.

The following neighbor query for quick dog matches two documents containing quick and dog, but gives a higher score for the document closer to quick and dog:

post/my_index/my_type/_search{ "query" : { "match_phrase" : { "title" : { "query" : "quick dog" , "slop" : -}      }   }}

{ "hits" : [     { "_id" : "3" , "_score" :0.75, "_source" : { "title" : "Thequick brown fox jumps over the quick dog" }     },     { "_id" : "2" , "_score" :0.28347334, "_source" : { "title" : "Thequick brown fox jumps over the lazy dog" }     }  ]}

Use proximity to increase relevance

Although proximity queries (Proximity query) work, the requirement that all entries must appear in the document is overly restrictive. This question is similar to what we discussed in the section on precision control in full-text search (Full-text search) (controlling Precision): If there are 6 matches in 7 entries, then the document may be relevant to the user, but the match _phrase the query will exclude it.

As an absolute requirement for proximity matching, we can treat it as a signal (Signal)-as a member of a number of potential matches, contribute to the final score of each document (see the Multi-digit field).

The fact that we need to accumulate the scores of multiple queries means that we should use a bool query to merge them.

We can use a simple match query as a must clause. This query is used to determine which documents need to be included in the result set. Long tail can be removed by minimum_should_match parameters. We then add more specific queries in the form of a should clause. Each document that matches the should clause increases its relevance.

get/my_index/my_type/_search{ "query" : { "bool" : { "must" : { "match" : { "title" : { "query" : "quick brown fox" , "minimum_should_match" : "30%" }        }      }, "should" : { "match_phrase" : { "title" : { "query" : "quick brown fox" , "slop" : -}        }      }    }  }}

There is no doubt that we can add additional queries to the should clause, each of which is used to increase the relevance of a particular type.

[Elasticsearch] Proximity match (ii)-multivalued field, proximity and correlation