45 python distributed crawler build search engine Scrapy explaining-elasticsearch (search engine) BOOL combination query

BOOL Query Description

filter:[], field filtering, not participating in the scoring
Must:[], if you have multiple queries, you must satisfy the "and"
Should:[], if there are multiple queries that satisfy one or more of the matching "or"
Must_not:[], on the contrary, the query word is not satisfied with the "reverse, non-"

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, must meet "and" # "     should": [],      if there are multiple queries, satisfy one or more of the matching "or" #     "Must_not": [] ,    on the contrary, the query word is not satisfied with the match "inverse, not" #}

Set up test data

#建立测试数据POST jobbole/job/_bulk{"index": {"_id": 1}}{"Salary": Ten, "title": "Python"} {"index": {"_id": 2}}{"Salary": 20, " Title ":" Scrapy "} {" index ": {" _id ": 3}}{" Salary ":" title ":" Django "} {" index ": {" _id ": 4}}{" Salary ": +," title ":" Elasticsearch "}

BOOL Combination query-The simplest term query of filter query, equivalent to equal to

Filter query to Salary field equals 20 data

You can see the execution of two two steps, the first to find all the data, and then all the data found in the filtered query to the Salary field equals 20 of the data

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, you must meet the #     "should": [],      if there are multiple queries, satisfy one or more of the matching #     "Must_not": [],    On the contrary, the query word is not satisfied with the match #} #简单过滤查询 # simplest filter filter Query # If we want to check the salary field equals 20 data get jobbole/job/_search{  "query": {    "bool": {                   #bool组合查询      "must": {                  #如果有多个查询词 must satisfy        "Match_all": {}          #查询所有字段      },      "filter": {               #filter过滤        "term": {               #term查询, will not use our search terms to participle, the search term exactly match the query          "salary":          #查询salary字段值为20        }      }    }  }} #简单过滤查询 # simplest Filter Query # If we want to check the salary field equals 20 data get jobbole/job/_search{  "query": {"    bool": {      "must":        { "Match_all": {}      },      "filter": {        "term": {          "salary":    +}}}  }

BOOL Combination query-The simplest filter query terms query, equivalent to or

Filter queries to salary fields equal to 10 or 20 of data

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, you must meet the #     "should": [],      if there are multiple queries, satisfy one or more of the matching #     "Must_not": [],    On the contrary, the query word is not satisfied with the match #} #简单过滤查询 # simplest filter filter Query # If we want to check the salary field equals 20 of the data # Filter Salary field value is 10 or 20 of the data get jobbole/job/_search{  "Query": {"bool": {"must": {"      Match_all": {        }      },      "filter": {"        terms": {          " Salary ": [10,20]}}}}  

Note: Filter filters can also be used for other basic queries

_analyze test to see the results of the word breaker parsing
Analyzer set word breaker type Ik_max_word refinement word segmentation, ik_smart non-refined word segmentation
Text set Word

#_analyze测试查看分词器解析的结果 #analyzer Set the word breaker type Ik_max_word fine-grained word segmentation, ik_smart non-refined word segmentation #text set Word get _analyze{  "Analyzer": "Ik_ Max_word ",  " text ":" Python network development Engineer "}get _analyze{  " Analyzer ":" Ik_smart ",  " text ":" Python Network development Engineer "}

BOOL Combination query--combinatorial complex query 1
The Query salary field equals 20 or the title field equals Python, the salary field is not equal to 30, and the salary field does not equal 10 of the data

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, must meet "and" # "     should": [],      if there are multiple queries, satisfy one or more of the matching "or" #     "Must_not": [],    on the contrary, the query word is not satisfied with the "inverse, non-" #}# Query salary field equals 20 or the title field equals Python, salary field is not equal to 30, and the salary field is not equal to 10 of the data get Jobbole/job/_ search{  "Query": {"bool": {"      should": [{"term"        : {"salary": +}},        {"term": {"title": "Python" }}      ],      "Must_not": [        {"term": {"salary": +}},        {"term": {"Salary": Ten}]  }}}

BOOL Combination query--combinatorial complex query 2
The Query salary field equals 20 or the title field equals Python, the salary field is not equal to 30, and the salary field does not equal 10 of the data

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, must meet "and" # "     should": [],      if there are multiple queries, satisfy one or more of the matching "or" #     "Must_not": [],    on the contrary, the query word is not satisfied with the "inverse, non-" #}# query the title field equals Python, or, (the title field equals Elasticsearch and salary equals 30) the data get Jobbole/job/_ search{  "Query": {"bool": {"      should": [{"Term        ": {"title": "Python"}},        {"bool": {          "must ": [            {" term ": {" title ":" Elasticsearch "}},            {" term ": {" salary ":    +}}}  }

BOOL Combination query-filter empty and non-empty

#建立数据POST bbole/jo/_bulk{"index": {"_id": "1"}}{"tags": ["Search"]}{"index": {"_id": "2"}}{"tags": ["Search", "python" ]}{"index": {"_id": "3"}}{"Other_field": ["Some data"]}{"index": {"_id": "4"}}{"tags": null}{"index": {"_id": "1"}}{" Tags ": [" search ", NULL]}

Methods for handling NULL null values

Gets the tags field, the value is not empty, and the value is not NULL data

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, must meet "and" # "     should": [],      if there are multiple queries, satisfy one or more of the matching "or" #     "Must_not": [],    on the contrary, the query word is not satisfied with the match "inverse, not" #} #处理null空值的方法 # Gets the tags field, the value is not empty and the value is not null data get bbole/jo/_search{  "query": {    " BOOL ": {" filter ": {"        exists ": {          " field ":" Tags "  }}}}

Gets the tags field value is empty or null data, if the data does not have a tags field will also get

# BOOL Query # old version of filtered has been replaced by BOOL # with BOOL including must should must_not filter to complete the # format as follows: #bool: {#     "filter": [], the filter of the      field, Do not participate in the scoring #     "must": [],        if there are multiple queries, must meet "and" # "     should": [],      if there are multiple queries, satisfy one or more of the matching "or" #     "Must_not": [],    on the contrary, the query word is not satisfied with the match "inverse, non-" #} #获取tags字段值为空或者为null的数据, if the data does not have a tags field will also get get bbole/jo/_search{  "query": {    " BOOL ": {      " Must_not ": {"        exists ": {          " field ":" Tags "  }}}}

45 python distributed crawler build search engine Scrapy explaining-elasticsearch (search engine) BOOL combination query