Consul Environment Construction and testing

Consul is a distributed, highly available service software that supports multi-datacenter service discovery and configuration sharing, developed by Hashicorp company in Go language, and open source based on the Mozilla Public License 2.0 protocol. On the Consul documentation, CONSUL supports service Discovery, health Checking, Key/value Store, Multi DataCenter. Using Consul, you can build complex applications and service discoveries in the system.

Advantages of Consul

• Using the Raft algorithm to ensure consistency is more straightforward than a complex Paxos algorithm. In comparison, Zookeeper uses Paxos, while ETCD uses Raft.
• Support multi-data center, internal and external network services using different ports for monitoring. A multi-data center cluster avoids single points of failure in a single data center, and its deployment requires consideration of network latency, fragmentation, and so on. Zookeeper and ETCD do not provide support for multi-datacenter functions.
• Supports health checks. ETCD does not provide this functionality.
• Supports HTTP and DNS protocol interfaces. The integration of the zookeeper is complex and the ETCD only supports the HTTP protocol.
• Official web Management interface, Etcd no such function.

Consul Architecture

As you can see, Consul is a multi-datacenter support, and multi-datacenter is a very common user scenario. The adoption of consul in the company's projects this is a more important consideration, the previous project has been using zookeeper, to achieve a multi-data center will have more problems.

• Agent-Daemon,agent running on each node of the consul cluster can run on server or client, each agent can provide DNS or HTTP interface, responsible for health Check and service discovery, and so on. Use the Consul Agent command to start, specifically run "Consul agent--help" to view the parameters.

• The client-client forwards all the RPCs to the server side, which is relatively stateless. The only time the client side performs a LAN gossip pool while running in the background consumes very little resources and network bandwidth.
• Server-server is responsible for maintaining cluster state, RPC quiries, and other data centers for exchanging WAN gossip, forwarding queries to leader or other data centers.

• Datacenter-Here the document refers to the time when the network environment is private, low latency, high bandwidth, exclusion and public network communication. In fact, I think the concept is somewhat similar to that of VMware, a virtual machine under one of VMware's servers can also be defined as a datacenter, and different servers can form different datacenter, which also includes the partitioning of logical functions. For us testers, a single-person test environment can be used as a datacenter.

• Serf:consul is based on serf, serf is a service discovery, Orchestration (application cluster management, etc.) tool, it is centralized, highly available and can fail back (tolerate), serf use gossip protocol, written in go language. Serf provides the functions of membership, error checking, and broadcast. The communication between any node-to-node contained in the gossip is mainly based on UDP.

• LAN Gossip-refers to the LAN Gossip pool that contains the node that is on the same LAN or datacenter. which contains nodes that is all located on the same local area network or datacenter.

• Wan Gossip-refers to the WAN Gossip pool that contains node that is on the same LAN or datacenter.

• Rpc-remote Procedure call. Client and server-side request/response mechanisms.

In this document, I'll cover some basic uses of consul and use consul in production environments.

Consul test Environment

The first time to study consul, with vagrant built 4 virtual machines, there is a problem is not resolved. My vagrant virtual machine is created on Mac, eth0 is the network of Mac, Eth1 is the private networks we created. But consul default on the binding eth0, so that 4 machines can not be eth0 to communicate, consul have relevant parameters to modify the Bind IP and port, this will be introduced later in the article. So I'm still using the existing 4 CentOS VMware virtual machine tests.

Node        TypeServer1    serverserver2    serverserver2    serveragent1     Agent

installation of Consul on CentOS

安装非常简单，安装完成后，就可以run Consul命令啦。

cd/usr/local/bin wget https://dl.bintray.com/mitchellh/consul/0.5.0_linux_ Amd64.zip Unzip *. Zip RM *. Zip


consul命令行运行的方式非常简单，可以看到以下的参数，也可以参考consul的用户文档。

$ consulusage:consul [--version] [--help] <command> [<args>]available Commands are:agent Runs a Consul agent force-leave forces a member of the cluster to enter the" Left" StateInfoProvides debugging information foroperatorsJoinTell Consul Agent toJoincluster keygen generates a new encryption key leave Gracefully leaves the Consul cluster and S Huts Lists The members of a Consul cluster monitor Stream logs from a Consul agent rel Oad Triggers the agent to reload configuration files version Prints the Consul version

Turn on Bootstrap Server

At the beginning of learning Consul, we need to get consul up and running. Consul server recommended for at least three to five, the recommended method is to start one of the servers, and configure the mode to bootstrap, which node can designate itself as a leader instead of an election. Then start another server in turn, configured as a non-bootstrap mode. Finally, the first serverbootstrap mode is stopped and restarted in non-bootstrap mode, so that the leader can be automatically elected between servers.

In the table above, we need to specify server 1 as bootstrap server, you can run the following command, and the following is the startup process. You can see that the bootstrap mode configures itself as leader.

[Email protected] ~]$ Consul agent-server-bootstrap-data-dir/tmp/Consul==> warning:bootstrap Mode enabled!Do not enable unless necessary==> Warning:it is highly recommended to set gomaxprocs higher than1==>starting Consul agent ...==>starting Consul agent RPC ...==> Consul Agent running!Node Name:'Localhost.localdomain'Datacenter:'DC1'Server:true(Bootstrap:true) Client Addr:127.0.0.1(HTTP:8500, HTTPS:-1Dns:8600Rpc:8400) Cluster Addr:10.74.15.87(LAN:8301, WAN:8302) Gossip encrypt:false, Rpc-tls:false, tls-incoming:falseAtlas:<disabled> ==> Log data would now streaminchAs it occurs: -/ to/ One  Geneva: -: .[INFO] Raft:node at10.74.15.87:8300[Follower] entering Follower state -/ to/ One  Geneva: -: .[INFO] Serf:EventMemberJoin:localhost.localdomain10.74.15.87     -/ to/ One  Geneva: -: .[INFO] Serf:EventMemberJoin:localhost.localdomain.dc110.74.15.87     -/ to/ One  Geneva: -: .[INFO] consul:adding server Localhost.localdomain (Addr:10.74.15.87:8300) (DC:DC1) -/ to/ One  Geneva: -: .[INFO] consul:adding server LOCALHOST.LOCALDOMAIN.DC1 (Addr:10.74.15.87:8300) (DC:DC1) -/ to/ One  Geneva: -: .[ERR] agent:failed toSyncremote State:no cluster leader -/ to/ One  Geneva: -: ,[WARN] raft:heartbeat timeout reached, starting election -/ to/ One  Geneva: -: ,[INFO] Raft:node at10.74.15.87:8300[candidate] entering candidate state -/ to/ One  Geneva: -: ,[INFO] Raft:election won. Tally:1     -/ to/ One  Geneva: -: ,[INFO] Raft:node at10.74.15.87:8300[Leader] entering Leader state -/ to/ One  Geneva: -: ,[INFO] consul:cluster leadership acquired -/ to/ One  Geneva: -: ,[INFO] consul:new leader Elected:localhost.localdomain -/ to/ One  Geneva: -: ,[INFO] raft:disabling Enablesinglenode (bootstrap) -/ to/ One  Geneva: -: ,[INFO] Consul:member'Localhost.localdomain'joined, marking health alive -/ to/ One  Geneva: -: ,[INFO] agent:synced Service'Consul'

To start the other servers, in and server2 server3 , we start consul with the following command, without the bootstrap option.

[Email protected] ~]$ Consul agent-server-data-dir/tmp/Consul==> Warning:it is highly recommended to set gomaxprocs higher than1==>starting Consul agent ...==>starting Consul agent RPC ...==> Consul Agent running!Node Name:'Localhost.localdomain'Datacenter:'DC1'Server:true(Bootstrap:false) Client Addr:127.0.0.1(HTTP:8500, HTTPS:-1Dns:8600Rpc:8400) Cluster Addr:10.74.15.87(LAN:8301, WAN:8302) Gossip encrypt:false, Rpc-tls:false, tls-incoming:falseAtlas:<disabled> ==> Log data would now streaminchAs it occurs: -/ to/ One Geneva: on: Wu[INFO] Serf:EventMemberJoin:localhost.localdomain10.74.15.87     -/ to/ One Geneva: on: Wu[INFO] Serf:EventMemberJoin:localhost.localdomain.dc110.74.15.87     -/ to/ One Geneva: on: Wu[INFO] Raft:node at10.74.15.87:8300[Follower] entering Follower state -/ to/ One Geneva: on: Wu[INFO] consul:adding server Localhost.localdomain (Addr:10.74.15.87:8300) (DC:DC1) -/ to/ One Geneva: on: Wu[INFO] consul:adding server LOCALHOST.LOCALDOMAIN.DC1 (Addr:10.74.15.87:8300) (DC:DC1) -/ to/ One Geneva: on: Wu[ERR] agent:failed toSyncremote State:no cluster leader -/ to/ One Geneva: on: -[WARN] Raft:enablesinglenode disabled, and no known peers. Aborting election.

you can see from log that you cannot find cluster leader and cannot configure yourself for Leader,server2 and Server1 communication. at this point, you need to run the following command to join the Server1 cluster.

Join 192.0. 2.2 192.0. 2.3 Consul   members // View the existing cluster node below

When 3 machines are added to the cluster, we need to configure these 3 machines for the same server, and let them choose leader themselves, you can stop the first consul, and then use the command to start:

Consul agent-server-data-dir /tmp/join192.0. 2.2 192.0. 2.3

现在，3太机子会互相复制信息，或者可以处理某台机子挂掉后的情况。当有其他server需要加入时，重复上面的操作就可以。

Reference
http://blog.coding.net/blog/intro-consul?type=hot
http://liubin.org/2014/02/22/first-serf-experience/

Consul Environment Construction and testing