Go MicroServices-part fifth-deploying in Docker swarm

This is a creation in Article, where the information may have evolved or changed.

Part V: Go microservices-Deploying in Docker swarm

In this section, we let the Accountservice service run on a locally deployed Docker swarm cluster and explore the core concepts of container orchestration.

This article deals with the following:

• Docker Swarm and container orchestration.
• Use Docker to containerized the Accountservice service.
• Configure the local Docker swarm cluster.
• Deploy the Accountservice service in a swarm service way.
• Run benchmark tests and collect metrics.

In fact, this part is not directly related to the go language, specifically it is about Docker and Docker swarm. Also hope you enjoy this article.

What is Docker orchestration (CONTAINER ORCHESTRATOR)

It may be useful to quickly introduce the concept of container orchestration before you start a real operation.

As applications become more complex and need to handle higher loads, we have to deal with the fact that thousands of our service instances are spread across a large number of physical hardware. Container orchestration lets us treat all the hardware as a single logical entity.

Container Orchestration This article summarizes the following:

Container orchestration: By abstracting the host infrastructure, orchestration tools allow users to treat the entire cluster as a single deployment target.
Abstracting the host infrastructure, orchestration tools allow users to treat the entire cluster as a single deployment ta Rget.

I can't summarize it myself well-using a container orchestration, such as Kubernetes or Docker Swarm, allows us to deploy our software components as services on one or more available infrastructures. In the case of Docker-the Swarm mode is about managing a Docker engine cluster called Swarm. Kubernetes uses a slightly different term and a critical abstraction hierarchy, but generally the concept is roughly the same.

The container orchestration provides a mechanism for service discovery, load balancing, internal addressing, and logging, not only for us to process the service life cycle.

The core concepts in Docker swarm

There are three concepts to be introduced in Docker swarm:

• Node: The node is an instance of the Docker engine that participates in swarm. Technically, it can be viewed as a host with its own CPU resources, memory, and network interfaces. The node can be either a management node or a worker node.
• Service: A service is something that executes on a worker node, defined by a container image and a command that instructs the container to execute. The service can be either replicated or global. A service can be viewed as an abstraction that allows any number of containers to form a logical service that can be accessed within a cluster or outside the cluster without having to know the topology of the network within the environment by its name.
• tasks (Task e.g. container): For all practical purposes, you can think of a task as a Docker container. The Docker document definition task is something that hosts the Docker container and the commands that run inside the container. The Manager node assigns tasks to the (worker) nodes, which are container mirrors that are specified in a particular service.

Shows a possible (simplified) deployment of the microservices landscape, where two nodes run five container instances and are abstracted into two service accountservice and Quotes-service.

Source

This section does not make any changes to the go code, just adds some new files to run the service on Docker. Code address: Https://github.com/callistaen ....

Accountservice the container of the

Docker installation

Docker installation, refer to the official website installation instructions.

Create Dockerfile

Dockerfile is Docker used to build a docker container image, containing all the things you want to include. Let's start by creating a dockerfile below the/accountservice directory.

FROM iron/baseEXPOSE 6767ADD accountservice-linux-amd64 /ENTRYPOINT ["./accountservice-linux-amd64"]

Quick explanation:

• From: Defines the base image from which we are going to start building our own image source. Iron/base is an image that is ideal for running a go application.
• EXPOSE: Defines the reachable port number that we want to expose inside the Docker network.
• Add: Adds a file accountservice-linux-amd64 to the root directory of the container file system/.
• EntryPoint: Defines the executable file to run when Docker launches this image container.

Build for another CPU architecture/os

As you can see, we have added linux-amd64 to the file name. Of course we can use the name to specify the go language executable, but I like the convention of putting the OS and target CPU platform into the executable file name. When I wrote this blog, I was using Mac OS x. So if I just run go build directly under the Accountservice directory, I will generate a accountservice executable under the same directory. But such files are in the Docker container, and the underlying OS is Linux-based. Therefore, before we build, we need to set up some environment variables so that the compiler and linker know that we are building a different OS and CPU architecture, and here we have an example of Linux.

export GOOS=linuxgo build -o accountservice-linux-amd64export GOOS=darwin

The above uses the-O flag to produce an executable binary file. I usually write a small script to help me do something that I might need to do repeatedly.

Since both OS X and linux-based containers are running in the AMD64 CPU architecture, we don't need to set (and reset) Goarch environment variables. But if you build for a 32-bit OS build or ARM processor, you need to set the Goarch properly before you build it.

Create a Docker image

Then we can now build our first Docker image to include our executables. Enter the Accountservice parent directory, which should be $gopath/src/github.com/callistaenterprise/goblog.

When building Docker container images, we typically use [Prefix]/[name] naming conventions to label their names. I generally use our github username as a prefix, such as Eriklupander/myservicename. For this blog series, we use the Someprefix prefix. Under the project root directory (eg:./goblog), execute the following command to build a Docker image based on the dockerfile above.

The
• builds the ACCOUNTSERVICE-LINUX-AMD64 executable file in the Goblog/accountservice directory.
• to build the Docker image.

Build Docker image in Goblog directory Docker build-t someprefix/accountservice accountservice/sending Build context to Docker Daemon 1 4.34kBStep: From iron/baselatest:pulling from Iron/baseff3a5c916c92:pull complete43f18fea29ad:pull completedigest : sha256:1489e9c1536af14937ac7f975b8529cbe335672b7b87dae36c6b385d3e4020c0status:downloaded newer image for iron/ Base:latest---> B438fe7f76e9step 2/4: EXPOSE 6767---> Running in 4246258b66c1removing Intermediate container 4246 258B66C1---> 5113056caf24step 3/4: ADD accountservice-linux-amd64/add failed:stat/var/lib/docker/tmp/ Docker-builder076553391/accountservice-linux-amd64:no such file or Directoryappledemacbook-pro-2:goblog apple$ Docker build-t Someprefix/accountservice accountservice/sending Build context to Docker daemon 7.457MBStep: from IR On/base---> B438fe7f76e9step 2/4: EXPOSE 6767---> Using cache---> 5113056caf24step 3/4: ADD accountservice-l inux-amd64/---> 7a21b55920e3step 4/4: entrypoint ["./accountservice-linux-amd64 "]---> Running in 5b7115e2f89dremoving Intermediate container 5b7115e2f89d---> 3e23a4268533successfully built 3e23a4268533successfully tagged someprefix/accountservice:latest

Well, we're now. The local Docker image warehouse contains an image named Someprefix/accountservice. If we want to run multiple nodes or want to share our image, we can use Docker push to pull the mirror out of the host other than the host provided by our current Docker engine.

We can then run the image directly from the command line.

docker run --rm someprefix/accountserviceStarting accountserviceSeeded 100 fake accounts...2018/05/16 02:57:37 Starting HTTP service at 6767

Note, however, that the container is no longer running localhost on your host OS. It is now located on its own network context, and we cannot access it directly from our actual host operating system. Of course there is a way to fix it, but let's not go into it first, we'll set up the Docker swarm locally and deploy Accountservice.

We first use CTRL + C to terminate this running image.

Configuring a single-node Docker swarm cluster

One of the goals of this blog is that we want to run our microservices on container orchestration. For many of us, it generally means kubernetes or Docker Swarm. Of course there are other orchestrations, such as Apache Mesos and Apcera, but this article specifically focuses on Docker 1.13 's Docker swarm.

The tasks you need to do when configuring a single-node Docker swarm cluster on your development computer may depend on how the Docker itself is installed. It is recommended to follow Swarm Tutorial, or you can use my way to use Docker Toolbox, Oracle VirtualBox and Docker-machine, based on my colleague Magnus Lab-repo's article on Service discovery.

Installing VirtualBox

VirtualBox is an open-source virtual machine software. If you install Docker Swarm using a Windows or OS X system, you need to install this software.

Download Address: Http://download.virtualbox.or ...

Installing Boot2docker

Boot2docker is a lightweight Linux hairstyle package designed specifically for Docker to solve problems with Windows or OS X users who can't install Docker. The Boot2docker is fully running in memory, 24M in size, and starts only 5-6 seconds.

Download Address: Https://github.com/boot2docke ....

Create Swarm Manager

The Docker swarm cluster consists of at least one swarm manager and 0 to more swarm workers. For the sake of simplicity, we only use one Swarm manager-here at least one. After this section, it's important that you have swarm manager up and running.

For our example here, use Docker-machine and make a virtual Linux machine run in our Swarm manager's VirtualBox. Here we use "swarm-manager-1" to mark this manager. You can also refer to the official documentation to see how to create swarm.

We use the following command to initialize the Docker-machine host and to indicate that it is the IP address of the Swarm node that is swarm-manager-1 as the Swarm-manager-1 node.

docker $(docker-machine config swarm-manager-1) swarm init --advertise-addr $(docker-machine ip swarm-manager-1)

If we need to create a multi-node swarm cluster, we will ensure that the connection token generated by the above command is stored, and later if we need to add additional nodes to swarm.

Create an overlay network (create an OVERLAY networks)

Docker's overlay network is a mechanism used when we add "Accountservice" to swarm, so it can access other containers in the same swarm cluster without having to know the actual cluster topology.
Use the following command to create a network:

docker network create --driver overlay my_network

Here we named the network My_network.

Deploying Accountservice Services

Are almost ready, now let's start deploying our own "Accountservice" as a swarm service. The Docker Service create command receives many parameters, but it is not crazy. Here is our command to release "Accountservice":

docker service create --name=accountservice --replicas=1 --network=my_network -p=6767:6767 someprefix/accountservice

The following is a quick explanation of the parameters:

• Name: Give a logical name to our service. This name is also the name used by other services in the cluster for addressing the service. So another service wants to call Accountservice, and that service only needs to make a GET request: http://accountservice:6767/accounts/10001.
• Replicas: The number of instances we want to serve. If our Docker swarm cluster is multi-node, the Swarm engine will automatically distribute between these nodes.
• Network: Here we tell the service to bind the overlay network name we just created.
• P: Map, [internal Port number]:[external port number]. Here we use 6,767:6,767, but if we use 6,767:80, then we need to access the service by port number 80 when we call outside. Note that this is a mechanism that allows our services to be reachable from outside the cluster. Generally speaking, you do not need to expose the service. In general, we use a boundary server (for example, a reverse proxy) so that there are routing rules and there can be security configurations so that external consumers cannot reach your service unless you use a voluntary approach.
• Someprefix/accountservice: This parameter specifies the name of the image that we want the container to run. In our example, we specify the label when we create the container. Attention! If we are going to run a multi-node cluster, we need to push our images into the Docker repository, such as pushing to the public (free) Docker hub service. Of course we can also set up a private Docker warehouse, or pay to keep the mirror free.

That's it. Run the following command to see if our service has successfully started.

> docker service lsID            NAME               REPLICAS  IMAGE                      ntg3zsgb3f7a  accountservice     1/1       someprefix/accountservice

It's sweet. We can now use curl or a browser to query our API. The only thing you need to know is that the front is swarm public IP. Even though we ran only one instance for our service in the swarm of many nodes, overwriting the network and Docker swarm allowed us to request any swarm host to access the service based on the port number. In other words, two services cannot be exposed externally using the same port number. Their internal port numbers can be the same, but external must be unique.

In any case, remember the value of the environment variable Managerip we saved previously.

> echo $ManagerIP192.168.99.100

If the terminal session has been changed, you can re-export it.

export ManageIP=`docker-machine ip swarm-manager-0`

The following Curl Request API:

> curl $ManagerIP:6767/accounts/10000{"id":"10000","name":"Person_0"}

Very good, successful.

Deployment of a visualization tool

Using Docker's command-line API to check the status of swarm is completely feasible (Docker service LS), but using a more graphical rendering looks more interesting. For example, with the Docker Swarm Visualizer visualized, we can also deploy this Visual service tool in the same way as we did for the Accountservice service. So we can have another way to look at our cluster topology. It can also be used to ensure that we have a service exposed to a given port number in the cluster.

We can install visualizer from the pre-baked container image, one line of command:

docker service create \  --name=viz \  --publish=8080:8080/tcp \  --constraint=node.role==manager \  --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \  dockersamples/visualizer

This allows us to access it via Port 8000. Directly through http://$ManagerIP 8000, remember the value of the Managerip just now.

The Boiling content!

I also developed a small swarm visualizer, called Dvizz, that uses the go language, the Docker Remote API, and the d3.js to generate graphics. You can also use the following command to install the service:

docker service create \   --constraint node.role==manager \   --replicas 1 --name dvizz -p 6969:6969 \   --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \   --network my_network \   eriklupander/dvizz

Direct Access http://$ManagerIP: 6969, you can see similar shows.

It looks a little sloppy, but don't take it too seriously, it's still very interesting, look at the bounce, zoom the server up and down, and drag and drop each node.

Add other services, such as Quotes-service

The microservices sector now has only one service (our ubiquitous accountservice). Below we deploy a previously mentioned spring boot type of microservices "Quotes-service", the microservices I have placed in a public Docker repository and can be deployed directly.

docker service create --name=quotes-service --replicas=1 --network=my_network eriklupander/quotes-service

If you use Docker PS to enumerate the Docker containers that are running, we may see that it has started (or is starting).

> docker psCONTAINER ID    IMAGE                       COMMAND                 CREATED         STATUS                           PORTS                                           NAMES98867f3514a1    eriklupander/quotes-service "java -Djava.security"  12 seconds ago  

Note that here we do not export the port mappings for this service, meaning that it cannot be accessed from outside the cluster, only internally via internal port 8080. We will integrate this service in part VII of service discovery and load balancing.

If you have added the Dvizz service to the cluster, you will see a quotes-service in the graph.

copyall.sh Script

To simplify things, we can add a shell script to help us with the automated work of rebuilding and redeploying. Under the root directory of Goblog, create a copyall.sh file:

#!/bin/bashexport GOOS=linuxexport CGO_ENABLED=0cd accountservice;go get;go build -o accountservice-linux-amd64;echo built `pwd`;cd ..export GOOS=darwindocker build -t someprefix/accountservice accountservice/docker service rm accountservicedocker service create --name=accountservice --replicas=1 --network=my_network -p=6767:6767 someprefix/accountservice

This script sets the GOOS environment variable so that we can safely build a statically linked binary file for the LINUX/AMD64 schema and then run some Docker commands to rebuild the image file and redeploy it into the swarm service. You can save time and enter many letters less.

The build of the Go language project is not covered in depth in this article. Personally, I like the simplicity of the shell script, although I often use the Gradle plugin myself, and also know that a good ol make is quite popular.

Occupy Space and performance

From now on, all benchmark and collect cpu/memory metrics will be used for the services deployed in Docker swarm. This means that the result of the previous article is not comparable to the result of the current start.

CPU usage and memory usage will be collected using Docker stats, and we will use the Gatling test we used earlier.

If you run the load test yourself, the requirements introduced in the second section still apply. Note that you need to modify the IP of the BaseURL parameter to the Swarm manager node, for example:

mvn gatling:execute -Dusers=1000 -Dduration=30 -DbaseUrl=http://$ManagerIP:6767

Summarize

So far, we've learned how to launch the Docker swarm domain (only one node) locally, and how to package and deploy our Accountservice microservices as a Docker swarm service.

In the next section, we'll add a heartbeat check to our microservices.

Reference links

• Fifth part of English
• Dockerfile
• Gradle Plug-in
• Ol ' s make, GNU make
• Try new features in Docker
• Topic Home
• Next section