Using Docker to build Elasticsearch cluster __elasticsearch

1 Installation Environment

Install the multi-machine ES cluster (distributed cluster), install an ES node in three servers respectively, and these three nodes form an ES cluster. Because it is a small cluster, setting these three nodes can be a master node and a data node. The server's IP is 192.168.1.111, 192.168.1.112, and 192.168.1.113, respectively.

When installing a stand-alone ES cluster, install three ES nodes on a single server 192.168.1.114.

Elasticsearch 5.6 requires JDK 1.8 and above. The software version is shown below.

(1) JDK 1.8 (without additional installation)
(2) Docker 17.12.1-ce
(3) Elasticsearch 5.6.4 

2 Multi-machine ES cluster setup

Installed on the server 192.168.1.111, 192.168.1.112, and 192.168.1.113. 2.1 Node 1

On the 192.168.1.111.

1, pull the es of the Docker mirror image.

sudo docker pull elasticsearch:5.6.4

2. Create ES Data folder-data and configuration file-es.yml on the server, and set es.yml file contents. The details are described below.

#集群名

cluster.name:crm

#节点名

node.name:node-111

 

#设置绑定的ip地址, can be IPv4 or IPv6, the default is 0.0.0.0,

# Any IP network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds to this machine

, which is

automatically judged if it is not set

#值必须是个真实的ip地址 

network.publish_host:192.168.1.111

 

#设置对外服务的http端口, defaults to 9200

http.port:9200

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9300

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

Discovery.zen.ping.unicast.hosts: ["192.168.1.111:9300", "192.168.1.112:9300", "192.168.1.113:9300"]

The cluster has been functioning normally, with the fewest number of nodes eligible for master (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

discovery.zen.minimum_ Master_nodes:2

3. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

2.2 Node 2

On the 192.168.1.112.

1, pull the es of the Docker mirror image.

sudo docker pull elasticsearch:5.6.4

2. Create ES Data folder-data and configuration file-es.yml on the server, and set es.yml file contents. The details are described below.

#集群名

cluster.name:crm

#节点名

node.name:node-112

 

#设置绑定的ip地址, can be IPv4 or IPv6, the default is 0.0.0.0,

# Any IP network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds to this machine

, which is

automatically judged if it is not set

#值必须是个真实的ip地址  

network.publish_host:192.168.1.112

 

#设置对外服务的http端口, defaults to 9200

http.port:9200

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9300

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

Discovery.zen.ping.unicast.hosts: ["192.168.1.111:9300", "192.168.1.112:9300", "192.168.1.113:9300"]

The cluster has been functioning normally, with the fewest number of nodes eligible for master (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

discovery.zen.minimum_ Master_nodes:2

3. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

Docker run-d--name crm-es-p 9200:9200-p 9300:9300-v Config/elasticsearch.yml-v/data/crm/elasticsearch/data/data:/usr/share/elasticsearch/data elasticsearch:5.6.4

2.3 Node 3

On the 192.168.1.113.

1, pull the es of the Docker mirror image.

sudo docker pull elasticsearch:5.6.4

2. Create ES Data folder-data and configuration file-es.yml on the server, and set es.yml file contents. The details are described below.

#集群名

cluster.name:crm

#节点名

node.name:node-113

 

#设置绑定的ip地址, can be IPv4 or IPv6, the default is 0.0.0.0,

# Any IP network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds to this machine

, which is

automatically judged if it is not set

#值必须是个真实的ip地址  

network.publish_host:192.168.1.113

 

#设置对外服务的http端口, defaults to 9200

http.port:9200

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9300

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

Discovery.zen.ping.unicast.hosts: ["192.168.1.111:9300", "192.168.1.112:9300", "192.168.1.113:9300"]

The cluster has been functioning normally, with the fewest number of nodes eligible for master (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

discovery.zen.minimum_ Master_nodes:2

3. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

Docker run-d--name crm-es-p 9200:9200-p 9300:9300-v Config/elasticsearch.yml-v/data/crm/elasticsearch/data/data:/usr/share/elasticsearch/data elasticsearch:5.6.4

3 single Machine es cluster build

Installed on the 192.168.1.114. Pulls the Docker mirror of the ES.

sudo docker pull elasticsearch:5.6.4

3.1 Node 1

On the 192.168.1.114.

1. Create ES Data folder-data-1 and configuration file-es-1.yml on the server, and set es-1.yml file contents. The details are described below.

#集群名

cluster.name:erp

#节点名

node.name:node-114-1

 

#设置绑定的ip地址, can be IPv4 or IPv6, the default is 0.0.0.0,

# Any IP network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds to this machine

, which is

automatically judged if it is not set

#值必须是个真实的ip地址  

network.publish_host:192.168.1.114

 

#设置对外服务的http端口, defaults to 9200

http.port:9200

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9300

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

Discovery.zen.ping.unicast.hosts: ["erp-es-1:9300", "erp-es-2:9301", "erp-es-3:9302"]

#集群中一直正常运行的, The minimum number of nodes to qualify as Master node (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

Discovery.zen.minimum_master_ Nodes:2

2. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

3.2 Node 2

On the 192.168.1.114.

1. Create ES Data folder-data-2 and configuration file-es-2.yml on the server, and set es-2.yml file contents. The details are described below.

#集群名

cluster.name:erp

#节点名

node.name:node-114-2

 

#设置绑定的ip地址, can be IPv4 or IPv6, The default is 0.0.0.0, which refers to any IP

network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds the machine

, and if not set it automatically determines that the value must be a real IP address

network.publish_host:192.168.1.114

 

#设置对外服务的http端口, defaults to 9200

http.port:9201

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9301

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

discovery.zen.ping.unicast.hosts:["erp-es-1:9300", "erp-es-2:9301", "erp-es-3:9302"]

#集群中一直正常运行的, The minimum number of nodes to qualify as Master node (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

Discovery.zen.minimum_master_ Nodes:2

2. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

Docker run-d--name erp-es-2-P 9201:9201-p 9301:9301--link erp-es-1:erp-es-1-v/data/erp/elasticsearch/conf/es-2.yml: /usr/share/elasticsearch/config/elasticsearch.yml-v/data/erp/elasticsearch/data/data-2:/usr/share/ Elasticsearch/data elasticsearch:5.6.4

3.3 Node 3

On the 192.1.68.1.114.

1. Create ES Data folder-data-3 and configuration file-es-3.yml on the server, and set es-3.yml file contents. The details are described below.

#集群名

cluster.name:erp

#节点名

node.name:node-114-3

 

#设置绑定的ip地址, can be IPv4 or IPv6, The default is 0.0.0.0, which refers to any IP

network.bind_host:0.0.0.0 #设置其它节点和该节点交互的ip地址 that binds the machine

, and if not set it automatically determines that the value must be a real IP address

network.publish_host:192.168.1.114

 

#设置对外服务的http端口, defaults to 9200

http.port:9202

#设置节点之间交互的tcp端口, default is 9300

transport.tcp.port:9302

 

#是否允许跨域REST请求

http.cors.enabled:true

#允许 where

REST requests come from Http.cors.allow-origin: "*"

 

#节点角色设置

node.master:true

node.data:true

 

#有成为主节点资格的节点列表

discovery.zen.ping.unicast.hosts:["erp-es-1:9300", "erp-es-2:9301", "erp-es-3:9302"]

#集群中一直正常运行的, The minimum number of nodes to qualify as Master node (default 1)

# (Totalnumber of master-eligible NODES/2 + 1)

Discovery.zen.minimum_master_ Nodes:2

2. Create and run ES containers

Name the container, set the mapping port, and map the ES container's configuration file and data file to the server physical machine.

Docker run-d--name erp-es-3-P 9202:9202-p 9302:9302--link erp-es-1:erp-es-1--link erp-es-2:erp-es-2-v/data/erp/ela Sticsearch/conf/es-3.yml:/usr/share/elasticsearch/config/elasticsearch.yml-v/data/erp/elasticsearch/data/ Data-3:/usr/share/elasticsearch/data elasticsearch:5.6.4

4 View cluster through head

The head can be installed on only one server, which is installed on the 192.168.1.111 server. The installation steps are described below.

1, pull head image

sudo docker pull Mobz/elasticsearch-head:5

2, create and run the Head container, the command is as follows.

Docker run--name elasticsearch-head-5-P 9100:9100 mobz/elasticsearch-head:5  

3, open the head interface to view the cluster information. Http://192.168.1.111:9100/ 5 other 5.1. Install IK Chinese participle

If necessary, you can install IK Chinese word breakers for es. The installation steps are described below.

Enter the Docker container of the ES node, then go to the corresponding folder (plugins), and then download and extract the corresponding version of the IK plugin.

$ sudo docker exec-it  container ID (or container name)  /bin/bash 

$ cd plugins

$ wget https://github.com/medcl/ Elasticsearch-analysis-ik/releases/download/v5.6.4/elasticsearch-analysis-ik-5.6.4.zip 

$ unzip Elasticsearch-analysis-ik-5.6.4.zip 

The IK word breaker will not take effect until after the installation of the Docker container that requires the restart of the ES node. In the cluster state, each node needs to install an IK word breaker separately.

Attention:

(1) Note To install the IK word breaker that matches the ES version.

(2) The following is a different installation method, but it may be an error when restarting ES, causing the installation of IK to fail. Therefore, it is recommended that the previous installation method.

./bin/elasticsearch-plugin

The error is as follows:

Java.io.filenotfoundexception:/usr/share/elasticsearch/config/analysis-ik/ikanalyzer.cfg.xml (No such file Ordirectory) 

(3) The effect of Word segmentation verification reference GitHub:

Https://github.com/medcl/elasticsearch-analysis-ik/blob/v5.6.4/README.md

5.2. Install Docker-ui

Docker-ui is a visual management tool for Docker containers.

Open the link http://192.168.1.111:9000/and view the Docker of the 111 machine. 5.3. Es start error 1

1, Start ES error, errors described below.

ERROR:[1] Bootstrap checks failed 

[1]: maxvirtual memory areas [Vm.max_map_count] 65530 Low, istoo to at increase t [262144] 
 

2, the solution:

Elasticsearch uses Niofs and MMAPFS blends for a variety of files. Ensure that the maximum mapping count is configured so that sufficient virtual memory is available for mmapped files.

(1) Switch to root user modification configuration sysctl.conf 

vi/etc/sysctl.conf 

(2) Add the following configuration 

vm.max_map_count=655300 

(3) and execute command 

 Sysctl-p 

(4) Reboot Elasticsearch to start successfully. To 

see if Vm.max_map_count is set up successfully:

Cat/proc/sys/vm/max_map_count

5.4. Es start error 2

1, the problem

Cluster startup is a possibility to report the following error.

Java.net.NoRouteToHostException:No route to host in

Sun.nio.ch.SocketChannelImpl.checkConnect (Native method)

 ...

2. Solutions

(1) View the IP address of the Docker

(2) Add the Docker search IP address to the whitelist. Examples are as follows.

Firewall-cmd--permanent--zone=trusted--add-source=172.17.0.0/16

firewall-cmd--reload

5.5. Docker Container Interconnect

Interconnect multiple Docker containers on the same host. The--link parameters of the Dockerrun command allow the container to interact through a connection (linking) system. The parameters are formatted as follows.

--link Name:alias

Name is the name of the container you want to link to, and alias is the nickname for this connection. The container's connection system creates a safe tunnel between the containers based on the name of the container. 6 Reference Documents

(1) Click to open the link

(2) Click to open the link

(3) Click to open the link