Summary of the use of Elasticsearch

Recently, there is a business demand, that is, the Full-text search keyword query list, thus turning to the learning of ES, but also studied for most of the month, make a note, summed up their learning process.

Learning a new technology alone, it is always inevitable to go a lot of detours, this recommendation under the Basic course of ES, right, to study the official website tutorial on it.

1) Elasticsearch: Authoritative guide

Https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html

The authoritative Chinese course, for the English poor classmate, read this tutorial can quickly get started.

2) Elasticsearchreference official website English Course

Https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html

Really want to understand the ES or read more English official website documents, Knowledge point is the most comprehensive. ES version Selection

When it comes to ES version selection, beginners es, Most of the information found is for the 2.x version, this is intended to be used in the 2.x version, but read the es5.x version of the new feature description, the decision to choose the 5.x version, because the new version of the ES performance than the 2.x version of good too much, but also want to use Spring-data-elasticsearch, but SPR Ing-data does not support the 5.x version, and ES development is strong, the 5.5.0 version used half a month ago, has now been updated to 5.5.1. ES client Request Method

1 Java API: Create transportclient, recommend use of complex applications

2 Java REST Client: Creating Restclient

3 HTTP RESTful API: Access with the most original HTTP request

Currently temporarily used the third, the reason ES business needs single, do not need to dynamically create, delete index, easy to use, only need to learn ES rest grammar on it. In fact, later can switch to Restclient, it is a persistent HTTP link (use httpclient also need an HTTP connection pool), features such as the official said:

The low-level client ' s features include:

Minimal dependencies load balancing across all available nodes failover in case of node failures and upon specific E Codes failed Connection penalization (whether a failed node is retrieddepends on how many consecutive the times it failed t He more failed attempts Thelonger the client would wait before trying that same node again) persistent Connections Trace Lo Gging of requests and responses optional automatic discovery of cluster nodes

es common Plug-ins

1) Head Plugin

5.5 Use Tutorial: http://www.cnblogs.com/xing901022/p/6030296.html

2 IK Chinese analyzer-Chinese participle prerequisite, you can customize the dictionary

GitHub Address: Https://github.com/medcl/elasticsearch-analysis-ik

Plugin Download Address: https://github.com/medcl/elasticsearch-analysis-ik/releases Bo Master Update is timely, 5.5.1 already have.

3) Pinyin Analyzer

GitHub Address: Https://github.com/medcl/elasticsearch-analysis-pinyin

IK and pinyin the same author, founder of Elastic Chinese community.

4) Elasticsearch-analysis-lc-pinyin Analyzer

This plug-in is also very good, but no pinyin momentum big. Support full spell, first letter, Chinese mixed search. After the phonetic Full-text search is ready to test the effect, the current parser is using IK and pinyin. es cluster

ES cluster configuration, the authoritative tutorial is very rough, it took several days to return to the test, and finally found that the configuration file parameters did not thoroughly.

Minimum Master Nodes

Setting the minimum number of principal nodes is very important to the stability of the cluster. This setting is helpful for preventing brain crack, that is, there are two master in a cluster.

This configuration is to tell Elasticsearch that unless there is sufficient available master candidate nodes, no master will be elected, and only the master candidate nodes that are available will be elected.

This setting should always be configured with the number of quorum nodes eligible for the primary node, the number of legal nodes: (Number of nodes eligible for the primary node/2) +1. For example:

1, if you have 10 number of nodes that meet the rule, the legal number is 6.

2, if you have 3 candidate master, and 100 data nodes, the legal number is 2, you only need to calculate the number of nodes that have the primary node eligibility.

3. If you have 2 nodes that meet the rules, the number of legal nodes should be 2, but this means that if a node dog is brought, your entire cluster will not be available. Set to 1 will guarantee the function of the cluster, but it will not prevent the brain crack. Based on this situation, the best solution is to have at least 3 nodes.

Small clusters or local tests do not have to distinguish master node,data node,client node. However, to ensure maximum scalability, the production environment is officially recommended to differentiate between different types of nodes, and the default elasticsearch is both master node and data node. On the node of knowledge, can refer to the reprinted "Elasticsearch node Type".

I am currently using the cluster configuration: A client node, 3 Master/data mixed nodes. Use Restclient to save a client node. Create an index, type sample

1. Create INDEX and Configuration Analyzer

{"Settings": {"index": {"
            number_of_shards": 3,
            "Number_of_replicas": 1,
            "Analysis": {
                " Analyzer ": {"
                    Ik_analyzer ": {"
                        type ":" Custom ",
                        " Tokenizer ":" Ik_smart "
                    },
                    " Pinyin_analyzer ": {
                        "Tokenizer": "My_pinyin"
                    }
                },
                "Tokenizer": {
                    "My_pinyin": {"
                        type": "Pinyin",
                        ' keep_original ': True
                    }
                }
    }

2. Create type and set mapping

{' Producttour ': {' properties ': {' CompanyID ': {' type ': ' Integer '}, ' ProductCode ': {' type ': ' keyword '}, ' ProductType ': {' type
                        ":" "Text", "Analyzer": "Ik_analyzer", "fields": {"Pinyin": {
            ' Type ': ' Text ', ' Analyzer ': ' Pinyin_analyzer '}}
                }, "Gtype": {"type": "Keyword"}, "Linetype": {
            ' type ': ' keyword '}, ' productstate ': {' type ': ' Boolean '}, ' Auditstate ': {' type ': ' Integer '}, ' Productmaintitle ': {' type ': ' t
                        Ext ", Analyzer:" Ik_analyzer "," fields ": {" Pinyin ": {
 ' Type ': ' Text ',                       "Analyzer": "Pinyin_analyzer"}}}, "PR
                    Oductsubtitle ": {" type ":" Text "," Analyzer ":" Ik_analyzer "," fields ": {
                    "Pinyin": {"type": "Text", "Analyzer": "Pinyin_analyzer"
            }}, "Supplyproductname": {"type": "Keyword"
                }, "Productmainpic": {"type": "Keyword"}, "Productpic": {
            ' type ': ' keyword '}, ' DPT ': {' type ': ' keyword '},
                    ' arr ': {' type ': ' Text ', ' Analyzer ': ' Ik_analyzer ', ' fields ': {
                    "Pinyin": {"type": "Text", "Analyzer": "Pinyin_analyzer"
              }  }, "Productfeatures": {"type": "Text", "Analyzer": "Ik_analyzer"
                        , "Fields": {"Pinyin": {"type": "Text",
                ' Analyzer ': ' Pinyin_analyzer '}}}, ' Tripday ': { ' type ': ' Integer '}, ' Tripnight ': {' type ': ' Integer '}, ' Advancedays ': {' type ': ' Integer '}, ' Auditresult ': {' type ': ' keyword '}, ' Createtime ': {' type ': ' Date '}}}

3. Full-Text Search

{
    "from": 0,
    "size":  //Paging query "query
    ": {"bool": {"must": {"
                    multi_match": {   Full-Text Search
                        "query": "1st",  //Keyword
                        "fields": [   //Full-Text search field
                            "ProductType",
                            "Productmaintitle",
                            "Productsubtitle",
                            "arr",
                            "Productfeatures"
                    }
                }
            ],
            "filter": [   Filter criteria
                {
                    "term": {
                        "ProductType": "Themt"
        }
}}}}

If the keyword is a mixed Chinese character, the Full-text search field is replaced by:

"Fields": [
    //Full-Text search field "Producttype.pinyin",
    "Productmaintitle.pinyin",
    "Productsubtitle.pinyin"
    , "Arr.pinyin",
    "Productfeatures.pinyin"
]

Pits encountered in use:

Mapping set to strict, dynamic introduction of [Doc] is not available, using term type, that is, exact query, character type uses keyword. Cannot use text; If keyword is Chinese character, use IK word breaker; If the keyword is mixed with pinyin or pinyin, use pinyin word breaker;