Small and medium Team floor configuration Center detailed

Don't know when the configuration file was last modified, what was modified? Change the configuration file to republish the project or manually trigger the restart service? Do you find the configuration file wrong for no reason to affect normal deployment on line? Are you troubled by these problems? 50+ Online project, hundreds of + profile, we are often abused by these profiles are not love, it is time to make a change! This article will take you to solve these problems, drink coffee easy operation dimension

Configuration Center Selection

The principle of selection: simple, easy landing, do not pick the platform, do not pick the language, as little reliance.

Compared with the disconf, Apollo and other programs, the final choice of ETCD+CONFD scheme, basically in line with the above principle, and etcd we deployed kubernetes in the time has been used, considered pro.

Configuring the central architecture diagram

• The whole configuration Center adopts C/s mode, uses ETCD as server to store data, CONFD as client to ETCD data update
• For more convenient management wrote WebUI, is actually a ETCD service WebUI, mainly interacts with ETCD service, go ETCD Access data
• CONFD pull data from the ETCD cluster based on the configuration file, and then generate the final profile based on the template file's fixed location where the data is populated in the format you set
• Configuration files can also be check_cmd reload_cmd checked and reloaded by mates and commands after the profile is generated

Configuration Center Deployment Etcd Cluster

• System environment
  • System:debian 8
  • etcd:v3.3.9
• Server address
  • 192.168.107.101
  • 192.168.107.102
  • 192.168.107.103

All servers need to execute the following command to install ETCD and create directories

1. Download the ETCD installation package and unzip it

2. Copy the program to/usr/bin directory for easy execution, ETCD for go writing, direct operation, there are mainly two files Etcd and Ectdctl,

# mv etcd-v3.3.9-linux-amd64/etcd* /usr/bin/

3. Create ETCD Profile directory /etc/etcd and data storage directory/home/data/etcd

# mkdir /etc/etcd /home/data/etcd

The three node ETCD configuration files are as follows

Node1 Configuration

# cat /etc/etcd/etcd.conf name: 'node1'data-dir: /home/data/etcdlisten-peer-urls: http://192.168.107.101:2380listen-client-urls: http://192.168.107.101:2379,http://127.0.0.1:2379initial-cluster-state: 'new'initial-cluster-token: 'etcd-cluster-conf'advertise-client-urls: http://192.168.107.101:2379initial-advertise-peer-urls: http://192.168.107.101:2380initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

Node2 Configuration

# cat /etc/etcd/etcd.conf name: 'node2'data-dir: /home/data/etcdlisten-peer-urls: http://192.168.107.102:2380listen-client-urls: http://192.168.107.102:2379,http://127.0.0.1:2379initial-cluster-state: 'new'initial-cluster-token: 'etcd-cluster-conf'advertise-client-urls: http://192.168.107.102:2379initial-advertise-peer-urls: http://192.168.107.102:2380initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

NODE3 Configuration

# cat /etc/etcd/etcd.conf name: 'node3'data-dir: /home/data/etcdlisten-peer-urls: http://192.168.107.103:2380listen-client-urls: http://192.168.107.103:2379,http://127.0.0.1:2379initial-cluster-state: 'new'initial-cluster-token: 'etcd-cluster-conf'advertise-client-urls: http://192.168.107.103:2379initial-advertise-peer-urls: http://192.168.107.103:2380initial-cluster: node1=http://192.168.107.101:2380,node2=http://192.168.107.102:2380,node3=http://192.168.107.103:2380

Start after each node configuration is complete

Need to run in the background, recommended screen tools

After all three nodes have been started, the etcdctl member list list of clusters can be viewed by command to confirm the cluster status

# etcdctl member list732ca490026f580d: name=node3 peerURLs=http://192.168.107.103:2380 clientURLs=http://192.168.107.103:2379 isLeader=falsebc16d35c3ad1c5ee: name=node2 peerURLs=http://192.168.107.102:2380 clientURLs=http://192.168.107.102:2379 isLeader=truef7a043d3b65cd4a4: name=node1 peerURLs=http://192.168.107.101:2380 clientURLs=http://192.168.107.101:2379 isLeader=false

Confd

1. Download the CONFD and put /usr/bin/ it in the directory for easy use

# wget github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-amd64# mv confd-0.16.0-linux-amd64 /usr/bin/confd# chmod +x /usr/bin/confd

2. Create a new CONFD configuration file directory

# mkdir /etc/confd/{conf.d,templates}

3. Create a new resource file, the .toml end of the file has become a fixed format

# cat /etc/confd/conf.d/nginx.conf.toml [template]src = "nginx.conf.tmpl"dest = "/tmp/nginx.conf"keys = [   "/conf/project/env/nginx/nginx.conf",]check_cmd = "/usr/sbin/nginx -t -c {{.src}}"reload_cmd = "/usr/sbin/service nginx reload"

Here we create a new Nginx configuration resource file, parameter explanation:

• src: Specifies the location of the template file, which is the location of the Nginx profile template Tmpl
• dest: Specifies the absolute path of the final generated or updated configuration file, here in order to test our assignment to/tmp/
• keys: The key used inside the template file, which is the key for the project configuration file in the ETCD
• check_cmd: Check command executed after updating configuration file, here we check the Nginx config file for syntax error

• reload_cmd: After check pass can execute the command configured here, the previous check is not a problem, will execute the reload command reload the configuration file

• prefix: To configure the key prefix, for example, our key is to start with/conf, then you can add a configuration prefix="/conf" , in the bottom of the keys can be omitted out of/conf
• owner: Configure the user who generated the configuration file

• Mode: Configure permissions for the build configuration file

4. Create a new template file

# cat /etc/confd/templates/nginx.conf.tmpl {{getv "/conf/project/env/nginx/nginx.conf"}}

• CONFD template syntax has a lot, here do not repeat, specific to check the official website
• We are putting the contents of the entire configuration file as a value in the ETCD, so we just need a getv instruction to get the values of value to populate the target file.

Test of the joint adjustment

With the ETCD cluster and CONFD services deployed, we're going to test if they work together properly.

1. Create a new KV value on the ETCD server

# etcdctl set /conf/project/env/nginx/nginx.conf 'user  www-data;> worker_processes 4;> > pid        /var/run/nginx.pid;> error_log  /home/logs/nginx/error.log  warn;> > events  {>     use epoll;>     worker_connections 51200;> }> > http {>     default_type  application/octet-stream;> >     server {>         listen       80;>         server_name  domain.com;> >         root /home/project/webroot;>         index index.shtml index.html;>     }> }'

# 查看设置key的内容# etcdctl get /conf/project/env/nginx/nginx.confuser  www-data;worker_processes 4;pid        /var/run/nginx.pid;error_log  /home/logs/nginx/error.log  warn;events  {    use epoll;    worker_connections 51200;}http {    default_type  application/octet-stream;    server {        listen       80;        server_name  domain.com;        root /home/project/webroot;        index index.shtml index.html;    }}

• ETCD API Sub-v2 and V3 version, two versions of the difference is large, V3 optimized a lot, but consider compatibility and other issues we use the V2 version here
• The default is V2 version, you can switch to the V3 version through the environment variables export ETCDCTL_API=3 , v2 etcdctl -v can view the API version, V3 by etcdctl version viewing the API version

2. Start CONFD

# confd -watch -backend etcd -node=http://192.168.107.101:2379 -node=http://192.168.107.102:2379 -node=http://192.168.107.103:23792018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Backend set to etcd2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Starting confd2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Backend source(s) set to http://192.168.107.101:2379, http://192.168.107.102:2379, http://192.168.107.103:23792018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Target config /tmp/nginx.conf out of sync2018-08-23T13:46:13+08:00 onlinegame.i.nease.net confd[17084]: INFO Target config /tmp/nginx.conf has been updated

Configuration parameter Description

• -watch: Turn on Watch mode, listen to ETCD Configuration Center file changes, once there is a change here immediately update, without this option configuration Center modification client does not update
• -backend: Back-end type, currently supports ETCD, Zookeeper, Consul, Vault, Redis, file, rancher and many other types, CONFD also have a number of separate configuration for the backend type, specifically through confd --helpcommands to view

• -node: ETCD node address, there are multiple nodes, so write more than-node, we etcd is three nodes of the cluster so write here three times '-node '

• -onetime: Can be used to replace the above -watch parameters, indicating that the run once to exit, if you do not want to let the configuration file updates, just want to update, you may use this parameter

• -interval: Can be used to replace the upper -watch parameters, indicating how many seconds to backend to take the data, if you want to reduce ETCD server pressure, but also want to let the client configuration file update automatically, this parameter to control

3. Through the log above you can see that the/tmp/nginx.conf file has been synchronized and updated, view/tmp/nginx.conf to determine the correct content

WebUI Kerrigan

Can not all the configuration file updates through the command line way? For ease of management, it took three days (really three days) to write a WebUI, named Kerrigan, to implement the directory tree, online view configuration, modify configuration, view configuration update history and other practical features

Configure the page to configure the ETCD connection information

Home, left Item list (project information sync CMDB)

Click on the list of items, according to the corresponding rules to go to the ETCD inside out the directory structure is presented in tree form

Click on the profile, the right side will show the current profile content

Click on the "Edit" button to edit this configuration, create a new page, just edit not allowed to modify the path

Click on the "History" button, then jump to the history page of the profile, this page shows all the changes in the configuration file history

Written in the last

1. is not to say this interface ugly explode! No way, front end test plus line all I do, no cell design, so look at it, and the most important thing is not functional use it
2. Why not k8s the configmap? We originally wanted to use the k8s configmap to do the configuration center, but not all the projects are running in k8s, and modify the Configmap also need to restart the container to take effect, so there is no use of
3. ETCD Anyone can modify it, feel insecure ah? In fact, we are using the account password authentication, and only in the intranet, limit IP, security point, another solution is to ETCD to go SSL, but the client side to put the certificate is not the trouble to use
4. How do I confirm that the Client Profile update was successful? If you are a one-time startup you can determine whether the startup command is performing properly after starting the command, if you are watch mode or interval, then. It's just a human check, and I don't have a good way.

If you feel that the article is helpful to you, please forward sharing for more friends to see. If you don't feel like reading, read the following articles:

• Docker-based DevOps practices for small and medium teams
• Rapid construction of SQL automatic Audit system by small and medium teams