[Cluster build] record using Docker to build Elasticsearch cluster

This paper records the entire process of building elasticsearch clusters using Docker (the 2.1.2 examples used in this article), and process affinity is also applicable to elasticsearch2.x,5.x, and subsequent authors will continue to study es in depth, The next step is to make a retrofit test based on this cluster for source Elasticsearch (hereafter referred to as ES). 1. Environment Introduction

This paper runs the environment ubuntu16.04 + docker17.05 + official online download elasticsearch2.1.2, in addition Docker environment for Openjdk:8-jre-alpine. (The reason for using Alpine is that there are not too many unnecessary components and commands, Docker don't need too many components inside)
es source Download Address 1.1 Ubuntu

$ cat/proc/version
Linux version 4.4.0-46-generic (buildd@lcy01-10) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubunt u1~16.04.2) #67-ubuntu SMP Thu Oct 15:05:12 UTC 2016

1.2 Docker

$ docker-v
Docker version 17.05.0-ce, build 89658be

1.3 Elasticsearch (version ignored)

$./elasticsearch-version
version:2.1.2, build:c849dd1/2017-04-24t16:18:17z, jvm:1.8.0_101

2, the deployment process encountered problems 2.1 is mainly Docker network configuration, the default is bridging network, I do not know why multiple node startup can not be found after each other.

Solution: Modify the ELASTICSEARCH.YML configuration. 2.2 Docker automatically reassign IP each time the Docker instance is restarted.

Not resolved. 3. Deployment Process 3.1 Make the dockerfile that generates ES.

Refer to Dockerfile's git address , as shown in the following figure.

3.2 In the ES source file for the ES install head plugin.

$ plugin Install Mobz/elasticsearch-head

3.3 Docker mirroring of ES generation

The mirror name is psiitoy/elasticsearch:2.1.2.

$ docker build-t psiitoy/elasticsearch:2.1.2.

3.4 View existing bridge information.

Docker defaults to use DOCKER0 as a network bridge

$ brctl Show Bridge
name Bridge ID       STP enabled interfaces
Docker0     8000.0242552a7fb0   no      vetha061837

3.5 View the network bridge corresponding network segment address.

Network segment found to be 172.17.0.x

$ IP Addr Show Docker0
4:DOCKER0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU 1500 Qdisc noqueue State up Group Defau Lt 
    Link/ether 02:42:55:2a:7f:b0 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global Docker0
       valid_lft Forever Preferred_lft Forever
    inet6 fe80::42:55ff:fe2a:7fb0/64 scope link 
       valid_lft forever preferred_lft Forever

3.6 Modify the ELASTICSEARCH.YML configuration.

Avoid the Docker between the container ES services and the inability to detect each other.
The Problem of cluster (Elasticsearch) can not be detected by reference to some nodes

network.host:0.0.0.0
discovery.zen.ping.multicast.enabled:false
discovery.zen.ping_timeout:120s
Discovery.zen.minimum_master_nodes:2 #至少要发现集群可做master的节点数,
client.transport.ping_timeout:60s
Discovery.zen.ping.unicast.hosts: ["172.17.0.2", "172.17.0.3"]

3.7 Create Docker container X5.

Docker Run parameter Introduction:
When using the-v parameter, the representative mounts the local directory to the Mirror, where we can view the/usr/share/elasticsearch/logs directory in the container.
When the-d parameter is used, the container starts and goes backstage.

$ docker run-d-V ~/usr/share/elasticsearch/logs:/usr/share/elasticsearch/logs psiitoy/elasticsearch:2.1.2
66ca0270ef44
$ docker run-d-V ~/usr/share/elasticsearch/logs:/usr/share/elasticsearch/logs psiitoy/ elasticsearch:2.1.2
DDFBB7EF65BC

3.8 If you want to enter the container can perform Docker exec-it $CONTAINER _id

When using the-I parameter, open stdin for console interaction
When using the-t parameter, assign a TTY device that can support terminal logons, default to False

$ docker exec-it 66ca0270ef44/bin/bash
bash-4.3#

3.9 View cluster status

Curl Http://ip:port/_cat/health?v, you can also use Health?pretty
We saw that the cluster had 5 nodes joined, and the experiment was successful.

$ Curl http://172.17.0.2:9200/_cat/health?v
Epoch      timestamp cluster       status Node.total node.data PRI relo init unassign pending_tasks max_task_wait_time active_shards_percent  1497352537 11:15:37 Elasticsearch Green           5         5      0   0 0 0 0 0                  -                100% 

3.10 Test to create an index with the client

Create a number_of_shards for 8,number_of_replicas 1, called Twitter's index.
Refer to TestClient's git address

public class Clienttest {private static client client = Null;//client must be a single case, single case, single case.

    Do not construct multiple clients in the application. public static void Main (string[] args) {try {client = getclient ("Elasticsearch", "172.17.0.3", 9300) //client must be a single case, single case, single case.
            Do not construct multiple clients in the application.
            CreateIndex ("Twitter", "tweet");
        System.out.println ("#" + client);
        catch (Exception e) {e.printstacktrace ();
            Finally {if (client!= null) {client.close (); }}/** * Create ES client must be a single case, single case, single case.
     Do not construct multiple clients in the application. * ClusterName: Cluster name * NODEIP: The IP address of the node in the cluster * Nodeport: node's port * * @return * @throws unknownhostexceptio n */public static synchronized Client getclient (string clustername, string nodeip, int nodeport) throws Unknownh ostexception {//Set the cluster name settings settings = Settings.settingsbuilder (). put ("Cluster.name ", clustername). PuT ("Client.transport.sniff", false)//. Put ("Number_of_shards", 1)//. Put ("Number_of_replica

        S ", 0). Build (); Create the cluster client and add the cluster node address client client = Transportclient.builder (). settings (Settings). Build ()//. ADDT Ransportaddress (New Inetsockettransportaddress ("192.168.200.195", 9370))//. Addtransportaddress (New Inetso Ckettransportaddress ("192.168.200.196", 9370))//. Addtransportaddress (New Inetsockettransportaddress ("192.
                168.200.197 ", 9370))//. Addtransportaddress (New inetsockettransportaddress (" 192.168.200.198 ", 9370))
                                . addtransportaddress (New Inetsockettransportaddress (Inetaddress.getbyname (NODEIP),

        Nodeport));
    return client; /** * CREATE INDEX * NOTE: Notify owner of ES Cluster in production to create INDEX * @param indexname * @param DocumentType * @thr OWS IOException * * Private static void CreateIndex (String indexname, String DocumentType) throws IOException {final Indicesexistsresponse I
        Res = Client.admin (). Indices (). Prepareexists (IndexName). Execute (). Actionget ();
        if (ires.isexists ()) {client.admin (). Indices (). Preparedelete (IndexName). Execute (). Actionget (); } client.admin (). Indices (). Preparecreate (IndexName). Setsettings (Settings.settingsbuilder (). put ("Number_of_
        Shards ", 8). Put (" Number_of_replicas ", 1)). Execute (). Actionget ();                     Xcontentbuilder mapping = Jsonbuilder (). StartObject (). StartObject (DocumentType)// . StartObject ("_routing"). Field ("Path", "Tid"). Field ("Required", "true"). EndObject (). Startob Ject ("_source"). Field ("Enabled", "true"). EndObject (). StartObject ("_all"). Field ("Enabled", "false"). Endobj ECT (). StartObject ("Properties"). StartObject ("User"). Field ("Store", true. Field ("Type", "string"). Field ("Index", "not_analyzed"). EndObject ()
                . StartObject ("message"). Field ("Store", true). Field ("Type", "string")
                . Field ("Index", "analyzed"). Field ("Analyzer", "standard"). EndObject ()
                . StartObject ("Price"). Field ("Store", true). Field ("Type", "float") . EndObject (). StartObject ("Nv1"). Field ("Store", true). Field ("Type", "I
                Nteger "). Field (" Index "," no "). Field (" Null_value ", 0). EndObject ()
                . StartObject ("Nv2"). Field ("Store", true). Field ("Type", "integer")
                . Field ("Index", "not_analyzed"). Field ("Null_value",). EndObject ()
    . StartObject ("Tid")            . Field ("Store", true). Field ("Type", "string"). Field ("Index", "not_analyzed")                                    . EndObject ()//. StartObject ("location")//                                  . Field ("Store", true)//. Field ("Type", "Geo_point")//                                  . Field ("Lat_lon", true)//. Field ("Geohash", true)//                               . Field ("Geohash_prefix", true)//. Field ("Geohash_precision", 7)// . EndObject ()//. StartObject ("shape")//. Fiel D ("Store", true)//. Field ("Type", "Geo_shape")//. Fiel                                  D ("Geohash", true)//. Field ("Geohash_prefix", false)// . Field ("Geohash_precisiOn ", 7)//. EndObject (). StartObject (" Endtime "). Field (" Type "," date "). Field (" Store ", true). Field (" Index "," not_analyzed ")//2015-08 -21t08:35:13.890z. Field ("Format", "Yyyy-mm-dd hh:mm:ss| | Yyyy-mm-dd ' T ' HH:mm:ss.                                  Sssz "). EndObject (). StartObject (" date "). Field (" Type "," date ")//
                . Field ("Store", true)//. Field ("Index", "not_analyzed") 2015-08-21T08:35:13.890Z//. Field ("Format", "Yyyy-mm-dd hh:mm:ss| | Yyyy-mm-dd ' T ' HH:mm:ss. Sssz "). EndObject (). EndObject (). EndObject (). EndObject ()
        ;
                Client.admin (). Indices (). prepareputmapping (IndexName). SetType (DocumentType)
     . SetSource (mapping)           . Execute (). Actionget (); }


}

3.11 View Head cluster information, successful.

3.12 Stop and remove all Docker containers

Add-f parameter Force stop delete

$ docker Rm-f $ (Docker ps-aq) b1b17d0f41eb 55d9259ff921 ff4d42575213 66ca0270ef44 ddfbb7ef65bc