On-the-Go Micro-service Setup (v)-Deploying Docker Swarm

This is a creation in Article, where the information may have evolved or changed.

Title: Deploying Docker Swarm

In this section, we launch our Accountservice, which runs in a local Docker swarm cluster. Discuss several important concepts of container deployment at the same time
This blog is mainly about the following:

• Docker Swarm and container deployment
• Run Accountservice with Docker as a container
• Build a local Docker swarm cluster
• Deploying Accountservice as a swarm service
• Base Measurement and results

In fact, after writing this chapter, I found that this chapter is not related to go. But I hope you like it.

What is container deployment

A simple introduction to container deployment before the practice begins:
When an application becomes more complex and starts to have a higher load, hundreds of services will run on many hardware. Container deployment Let's manage our hardware on a single node
Summarized in an article:

abstract host infrastructure, deployment tools allow users to control the entire cluster on a single deployment target.

This concludes well. Use Kubernetes or Docker swarm this container deployment tool to deploy our various services on different nodes. For Docker, swarm mode management Docker Engine cluster. Kubernetes uses a slightly different abstract approach, but the overall concept is consistent.
Container deployment not only controls the life cycle of our services, but also provides other services such as service discovery, load balancing, internal addresses, and logs.

Docker Swarm Core Concepts

In Docker swarm, there are three core concepts:

• Node: A node is an instance of a Docker engine. In theory, he is a host with CPU resources, memory, and network interfaces. A node can be either a manager node or a worker node.
• Service: A service is a command that runs on a worker node. A service can be either replicated or global. A service can be abstracted as a logical service consisting of any number of containers. The service can be called with its name, without having to know its internal network structure.
• Task: A task can be a Docker container, and the Docker file definition task is to own and run the Docker container and instructions. The Manager node distributes the service to the worker node.

Demonstrate a simple microservices framework. Two services Accountservice and Quotes-service are abstracted into two nodes, running in five instances of containers.

! [Image of the crossing ...]

Code

This part does not change the go aspect code.

Cker. You can get a complete code for this chapter

git checkout P5

Container of our Accountservice

Docker installation

You need to install Docker. I use Docker Toolbox and Vitualbox. But you can use Docker directly.

Create Dockerfile

A dockerfile can be seen as a recipe for what kind of Docker image you want to create. Let's create a file in the Accountservice folder Dockerfile:

FROM iron/baseEXPOSE 6767ADD accountservice-linux-amd64 /ENTRYPOINT ["./accountservice-linux-amd64"]

Explain:

• from-defines our base image. We'll start with this image. Iron/base is a small image that can run the Go program
• expose-defines a port as a port for external requests
• add-Add a file accountservice-linux-amd64 to the root (/) directory
• entrypoint-defines which program to start when Docker opens this mirrored container

Compilation under different operating systems

Our file name contains LINUX-AMD64. We can call him any name, but I like to put the operating system and CPU model into the execution file name. I'm using the Mac OSX system. So if I just compile the go execution file, with Go build, This produces an execution file in the same folder. However, this executable file cannot be run on Docker because the Docker container environment is Linux. Therefore, we need to set some environment parameters so that our compiler knows we want to compile the files for other system or CPU environment.
Run under the Goblog/accountservice folder:

export GOOS=linuxgo build -o accountservice-linux-amd64export GOOS=darwin

-O indicates that a binary execution file is generated. I often do these things automatically with a script file (later)
Because OS X and Linux containers are both on AMD64 CPU architectures, we don't need to set the Goarch parameter. But if you use a 32-bit system, or an ARM processor, you want to set the Goarch parameter

Create a Docker image

Now we create the first Docker image, which contains our execution files. To the upper folder of Accountservice, is $gopath/src/github.com/callistaenterprise/goblog.
For Docker images, we often use a prefix to label names. I often use my github name as a prefix, such as eriklupander/myservicename. Here, I use Someprefix as a prefix. Execute the following command to create a docker image:

> docker build -t someprefix/accountservice accountservice/Sending build context to Docker daemon 13.17 MBStep 1/4 : FROM iron/base ---> b65946736b2cStep 2/4 : EXPOSE 6767 ---> Using cache ---> f1147fd9abcfStep 3/4 : ADD accountservice-linux-amd64 / ---> 0841289965c6Removing intermediate container db3176c5e1e1Step 4/4 : ENTRYPOINT ./accountservice-linux-amd64 ---> Running in f99a911fd551 ---> e5700191acf2Removing intermediate container f99a911fd551Successfully built e5700191acf2

All right, we got a someprefix/accountservice mirror. If we are going to run or share the image under multiple nodes, we can use Docker push to pull our mirror to another host.
We now run the image:

> docker run --rm someprefix/accountserviceStarting accountserviceSeeded 100 fake accounts...2017/02/05 11:52:01 Starting HTTP service at 6767

However, our container is not running under your host system and he is running on his own network and we cannot request him directly from our host. There are ways to solve this problem, but now let's put it in place and we continue to build our Docker swarm and deploy Accountservice.

Create a single-node Docker swarm cluster

A docker swarm cluster includes at least one swarm manager and 0 to more swarm workers. My example will include a swarm manager. After this section, you will have a swarm manager running.
You can refer to other articles to see how to run swarm. This command initializes the Docker host to swarm-manager-1 as a swarm node, while allowing the SWARM-MANAGER-1 node address to be the same as the host

> docker $(docker-machine config swarm-manager-1) swarm init --advertise-addr $(docker-machine ip swarm-manager-1)

If we were to create a multi-node swarm cluster, we would record the join-token generated by this command so that we could join the other nodes into the swarm.

Create a network

The use of a Docker network is that when we want to request other containers on the same swarm cluster, we don't need to know the actual cluster distribution.

docker network create --driver overlay my_network

My_network is our network name.

Deploying Accountservice

Now we're going to deploy our accountservice into the Docker swarm service. This Docker service command has many parameter settings, but don't be afraid. Here we deploy the Accountservice

docker service create --name=accountservice --replicas=1 --network=my_network -p=6767:6767 someprefix/accountservicentg3zsgb3f7ah4l90sfh43kud

Take a quick look at these parameters

• -name: The name of the service. This is also the name of the other services in the cluster that request us. So another service to request Accountservice, the service only needs a GET request http://accountservice:6767/accounts/10000
• -replicas: The number of instances we serve. If we have a multi-node Docker swarm cluster, swarm engine will automatically distribute the instances to different nodes.
• -network: Here we tell our service to use the network we just created My_network
• -P: Map [Internal Port]:[External Port]. Here we use 6767:6767. If we use 6,767:80, the external request takes 80 ports. Note that this section makes our service available to the outside world. In most cases, You should not expose your service to the outside world. You should use a edge-server (such as a reverse proxy), including a routing mechanism and security checks, so the outside world can't ask for your service
• Someprefix/accountservice: Indicates which image we want the container to run.

Let's see if our service is running.

> docker service ls

Great, we should be able to curl or request our API using our browser. The only thing to know is our swarm IP address. Even if we only run one service instance, our network and swarm will also need our service external ports, This means that two services cannot use the same external port. They can have the same internal port, but for the outside, swarm is a whole.

> echo $ManagerIP192.168.99.100

If you change the terminal, you can re-export:

> export ManagerIP=`docker-machine ip swarm-manager-0`

Curl Request:

> curl $ManagerIP:6767/accounts/10000{"id":"10000","name":"Person_0"}

Deployment visualization

Using Docker commands to view the status of swarm is not easy to see, a graphical approach is better. For example, Manomarks Docker swarm visualizer can be deployed as a Docker swarm service. This will give us a graph of our cluster distribution. , colleagues make sure that the services we expose externally in the cluster can be requested.
Initialize the visualizer in the container image

docker service create \ --name=viz \ --publish=8080:8000/tcp \ --constraint=node.role==manager \ --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ manomarks/visualizer

This will generate a service on port 8000. Let's browse with a browser http://$ManagerIP 8000

Additional content

I also made a swarm visual interface called Dvizz, which shows the Docker remote API and the D3.js force diagram. You can install her.

docker service create \ --constraint=node.role==manager \ --replicas 1 --name dvizz -p 6969:6969 \ --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \ --network my_network \ eriklupander/dvizz

Browse http: 6969/$ManagerIP

Join Quote-service

Only one service micro-service can not see the full picture of microservices. Let's deploy a spring boot-based quotes-service. I mirrored this container in the Eriklupander/quotes-service in the Docker hub.

> docker service create --name=quotes-service --replicas=1 --network=my_network eriklupander/quotes-service

If you enter Docker PS to see those docker containers running:

> docker psCONTAINER ID    IMAGE                       COMMAND                 CREATED         STATUS                           PORTS                                           NAMES98867f3514a1    eriklupander/quotes-service "java -Djava.security"  12 seconds ago  Up 10 seconds

Note that we do not expose an outside port to this service, so we can only request within the cluster on port 8080. We will integrate this service in the seventh part of the same time look at service exploration and load balancing.
If you join Dvizz, you should be able to see Quotes-service and Accountservice.

copyall.sh Script

Let's do a script to help us compile and deploy. In the Root/goblog folder, create a script file called copyall.sh

#!/bin/bashexport GOOS=linuxexport CGO_ENABLE=0cd accountservice; go get; go build -o accountservice-linux-amd64;echo build `pwd`;cd ..export GOOS=darwindocker build -t someprefix/accountservice accountservice/docker service rm accountservicedocker service create --name=accountservice --replicas=1 --network=my_network -p=6767:6767someprefix/accountservice

This script compiles the execution file, recompiling the Docker image, and deploys it to the Docker swarm service.
I like the simplification of scripts, although sometimes I use Gradle plugin.

Performance

Now, all the bases are on the Docker swarm. This means that the previous results cannot be used for comparison with the following
CPU usage and memory usage are collected using Docker stats. We also use Gatling testing.
If you like stress testing, the second section can still be used, but the-baseurl parameter needs to be changed

> mvn gatling:execute -dusers=1000 -Dduration=30 -DbaseUrl=http://$ManagerIP:6767

Memory utilization

> docker stats $(docker ps | awk '{if(NR>1) print $NF}')CONTAINER                                    CPU %               MEM USAGE / LIMIT    accountservice.1.k8vyt3dulvng9l6y4mj14ncw9   0.00%               5.621 MiB / 1.955 GiBquotes-service.1.h07fde0ejxru4pqwwgms9qt00   0.06%               293.9 MiB / 1.955 GiB

After launch, the Accountservice contains Linux and our container with 5.6MB of memory, Java developed Quotes-service 300MB. Although this can be reduced by tuning the JVM.

CPU and Memory use stress test

CONTAINER                                    CPU %               MEM USAGE / LIMIT   accountservice.1.k8vyt3dulvng9l6y4mj14ncw9   25.50%              35.15 MiB / 1.955 GiBB

Under 1K req/s, the swarm running in the virtual machine is slightly higher than the OS X system in section 23rd and the CPU is roughly the same.

Performance

! [Image of the crossing ...]

The delay rose to 4ms. This and the direct run have been raised for several reasons. I think the Gatling test has some delay in routing through bridged networks and swarm. But the 4ms delay is good. After all, we read the data from the BOLTDB, serialize it to JSON and output it to HTTP.

Summarize

We learn how to launch Docker swarm and deploy Accountservice to swarm. Next, we'll add Healthcheck to our microservices.