Docker Series seven (Duubo Micro Service Deployment practice).

First, preface

Prior to our company Deployment Services, which is what we all know (install JDK, tomcat-> compiled file or fight War package upload, start Tomcat), this kind of deployment has lasted for a long time, resulting in a lot of problems:

1, the heavy release task. MicroServices a lot, it is necessary to restart each service, and if the cluster, then the service to start more.

2, environmental migration error. Often happen one thing, the same set of code, this server is able to run up, change server is an error.

3, morale is low. Small companies do not have serious operations, are to let the development of mergers and acquisitions to do this work, and then responsible for this piece of colleagues complain a lot (because this deployment is too boring).

So the leader decided to get Docker as our way of deployment, which would be a good solution to the current project deployment problems, and the project to inject fresh blood.

From 15th last month to contact Docker, and now the micro-service architecture of our system initially set up, tossing a long time, stepped on a lot of pits. Commemorate a little achievement, wrote this blog. In order to avoid suspected leakage of company secrets, comprehensive rollup do some simple introduction ha, the following is the smallest micro-service architecture diagram, for example, the deployment of a set of Dubbo micro-services.

Second, the service image Package 1, the TOMCAT Foundation environment constructs

Every micro-service in our system is deployed to run on Tomcat (I heard that this is a bad way, for some not web engineering, no need to build Web services, add complexity, but also waste system resources), so my idea is: first set up a Tomcat environment image, Each microservices is then built on this environment image. So write a tomcat-env image, the idea is as follows:

-- JDK-based Tomcat container (main reference website tomcat image of Dockerfile).

-- in the context directory to hold the project compiled files, and rename to ROOT (the reason for not putting the war package is to consider debugging when convenient, do not change a file, a war package).

-- Delete the root file from the original Tomcat container WebApps directory and upload the root folder of the project in the context directory to the container WebApps directory.

-- start the service.

 fromOpenjdk:8-jreENVCatalina_home/usr/local/tomcatENVPATH $CATALINA _home/bin: $PATHRUNMkdir-p "$CATALINA _home"Workdir$CATALINA _home# Let"Tomcat Native"Live somewhere isolatedENVtomcat_native_libdir $CATALINA _home/native-jni-libENVld_library_path ${ld_library_path:+ $LD _library_path:} $TOMCAT _native_libdir# runtime dependencies for TOMCAT Native libraries# Tomcat Native1.2+ requires a newer version of OpenSSL than Debian:jessie has available# > checking OpenSSL library version >= 
   1.0.2... # > Configure:error:Your version of OpenSSL is not compatible with this version of the tcnative# see http:Tomcat.10.x6.nabble.com/vote-release-apache-tomcat-8-0-32-tp5046007p5046024.html (and following discussion) # and Github.com/docker-library/tomcat/pull/31ENVOpenssl_version 1.1.0f-3+DEB9U2RUNset-ex; \ currentversion="$ (dpkg-query--show--showformat ' ${version}\n ' OpenSSL)"; \ If dpkg--compare-versions"$currentVersion" << "$OPENSSL _version", then \ if! grep-q stretch/etc/apt/sources.list, then \# only add stretch if we' re not already building from within stretch{\ Echo' Deb Http://deb.debian.org/debian stretch main '; \ Echo' Deb http://security.debian.org stretch/updates main '; \ Echo' Deb Http://deb.debian.org/debian stretch-updates main '; \} >/etc/apt/sources.list.d/stretch.list; \ {\# Add a negative"Pin-priority"So, we never ever get packages from stretch unless we explicitly request them echo' Package: * '; \ Echo' Pin:release n=stretch* '; \ Echo' Pin-priority:-10 '; \ echo; \# except OpenSSL, which is the reason we' Re here echo ' Package:openssl libssl* '; \ Echo"Pin:version $OPENSSL _version"; \ Echo' pin-priority:990 '; \} >/etc/apt/preferences.d/stretch-openssl; \ fi; \ apt-get Update; \ apt-get install-y--no-install-recommends openssl="$OPENSSL _version"; \ rm-rf/var/lib/apt/lists/*; \ firun apt-get update && apt-get install-y--no-install-recommends \ libapr1 \ && RM -rf/var/lib/apt/lists/*# Seewww.apache.org/dist/tomcat/tomcat-$TOMCAT _major/keys# See also "update.sh" (github.com/docker-library/tomcat/blob/master/update.sh) ENV Gpg_keys 05ab33110949707c93a279e3d3efe6b686867ba6 07e48665a34dcafae522e5e6266191c37c037d42 47309207d818ffd8dcd3f83f1931d684307a10a5 541fbe7d8f78b25e055ddee13c370389288584e7 61B832AC2F1C5A90F0F9B00A1C506407564C17A3 713da88be50911535fe716f5208b0ab1d63011c7 79f7026c690baa50b92cd8b66a3ad3f4f22c4fed 9ba44c2621385cb966eba586f72c284d731fabee A27677289986db50844682f8acb77fc2e86e29ac A9c5df4d22e99998d9875a5110c01c5a2f6059e7 dcfd35e0bf8ca7344752de8b6fb21e8933c60243 F3a04c595db5b6a5f1eca43e3b7bbb100d811bbe F7da48bb64bcb84ecba7ee6935cd23c10d498e23env tomcat_major 8ENV tomcat_version 8.0.53ENV tomcat_sha512 cd8a4e48a629a2f2bb4ce6b101ebcce41da52b506064396ec1b2915c0b0d8d82123091242f2929a649bcd8b65ecf6cd1ab9c7d90ac0e261821097ab6f Be22df9env Tomcat_tgz_urls \#issues.apache.org/jira/browse/infra-8753?focusedcommentid=14735394#comment -14735394Www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz \# if the Version is outdated, we might has the to pull from the dist/archive:/www-us.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz \www.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz \archive.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gzenv tomcat_asc_urls \Www.apache.org/dyn/closer.cgi?action=download&filename=tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz.asc \# Not all the mirrors actually carry the. asc files: ' (www-us.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz.asc \www.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz.asc \archive.apache.org/dist/tomcat/tomcat-$TOMCAT _major/v$tomcat_version/bin/apache-tomcat-$TOMCAT _version.tar.gz.ascrun set-eux; \ \ savedaptmark= "$ (Apt-mark showmanual)"; \ apt-get Update; \ \ apt-get install-y--no-install-recommends GnuPG dirmngr; \ \ Export Gnupghome= "$ (mktemp-d)"; \ for key in $GPG _keys; Do \ gpg--keyserver ha.pool.sks-keyservers.net--recv-keys "$key"; \ done; \ \ apt-get install-y--no-install-recommends wget ca-certificates; \ \ success=; \ for URL in $TOMCAT _tgz_urls; Do \ If Wget-o tomcat.tar.gz "$url"; then \ Success=1; \ break; \ fi; \ done; \ [-N "$success"]; \ \ echo "$TOMCAT _sha512 *tomcat.tar.gz" | Sha512sum-c-; \ \ success=; \ for URL in $TOMCAT _asc_urls; Do \ If Wget-o tomcat.tar.gz.asc "$url"; then \ Success=1; \ break; \ fi; \ done; \ [-N "$success"]; \ \ gpg--batch--verify tomcat.tar.gz.asc tomcat.tar.gz; \ tar-xvf TomCat.tar.gz--strip-components=1; \ rm Bin/*.bat; \ rm tomcat.tar.gz*; \ command-v gpgconf && gpgconf--kill All | | :; \ rm-rf "$GNUPGHOME"; \ \ nativebuilddir= "$ (mktemp-d)"; \ tar-xvf bin/tomcat-native.tar.gz-c "$nativeBuildDir"--strip-components=1;        \ apt-get install-y--no-install-recommends \ dpkg-dev \ gcc \ libapr1-dev \ libssl-dev \ Make \ "openjdk-${java_version%%[.~bu-]*}-jdk= $JAVA _debian_version" \; \ (\ Export catalina_home= "$PWD"; \ CD "$nativeBuildDir/native"; \ gnuarch= "$ (dpkg-architecture- -query deb_build_gnu_type) "; \./configure \--build= "$gnuArch" \--libdir= "$TOMCAT _native_libdir" \--prefix= "            $CATALINA _home "\--with-apr=" $ (which apr-1-config) "\--with-java-home=" $ (docker-java-home) "\ --with-ssl=yes; \ make-j "$ (nproc)"; \ make install; \    ); \ rm-rf "$nativeBuildDIR "; \ rm bin/tomcat-native.tar.gz; \ \# Reset Apt-mark ' s "Manual" list So "purge--auto-remove" would remove all build dependencies Apt-mark auto ' . * ' >/dev/null; \ [-Z ' $savedAptMark '] | | Apt-mark manual $savedAptMark; \ apt-get purge-y--auto-remove-o apt::autoremove::recommendsimportant=false; \ rm-rf/var/lib/apt/lists/*;  \ \# SH Removes env vars it doesn ' t support (ones with periods) #github.com/docker-library/tomcat/issues/77find./bin/-name ' *.sh '-exec sed-ri ' s|^#!/bin/sh$|#!/usr/bin/env bash| ' ' {} ' +# verify Tomcat Native is working properlyrun set-e \ && nativelines= "$ (catalina.sh configtest 2>& ; 1) "\ && nativelines=" $ (echo "$nativeLines" | grep ' Apache Tomcat Native ') "\ && nativelines=" $ (ech O "$nativeLines" | sort-u) "\ && if! echo "$nativeLines" | grep ' info:loaded APR based Apache Tomcat Native library ' >&2; Then \ Echo >&2 "$nativeLines"; \ exit 1; \ fiexpose 8080RUN rm-rf/usr/local/tomcat/webapps/root/onbuild COPY root/usr/local/tomcat/webapps/root/onbuild ENTRY Point ["/usr/local/tomcat/bin/catalina.sh", "Run"]

tomcat-env

Looks very complicated, do not be frightened, actually are copy of official website Tomcat Mirror Dockerfile, and then changed a little, mainly the following three sentence: Delete the container root folder, copy the root folder of the context directory to the Wenapps directory, restart the service.

RUN rm-rf/usr/local/tomcat/webapps/root/ Onbuild COPY root/usr/local/tomcat/webapps/root/ Onbuild entrypoint ["/usr/local/tomcat/bin/catalina.sh", "Run"]

Tips: 1. Onbuild command This image will not be executed, only if the image is the base image.

2. The context directory refers to the directory where the Dockerfile file resides.

3, the image has been uploaded to DockerHub:hub.docker.com/r/jmcui/tomcat-env/

2, micro-service image Packaging

With the basic Environment image tomcat-env, packaging a service image is a simple thing to do:

 from tomcat-env:1.0

Yes, it is so simple, because we put all the work in the tomcat-env, is actually the effect of the Onbuild command ~ ~

Iii. Arrangement of documents Docker-compose.yml

Micro-service projects to be deployed, mainly by the Docker-compose.yml file orchestration, the relationship between services and the sequence of start-up, and then the dozens of scattered micro-services as a whole to unified management.

The first thing that bothers me is the network problem. As the developer knows, to specify (Spring in Applicationcontext.xml) database address and Zookeeper address in the project, how do I know what the container's IP address is? Let's get to know the network mode of Docker.

Docker's default network configuration is bridge, and when Docker starts, it automatically creates a Docker0 virtual bridge on the host, which is actually a bridge of Linux and can be understood as a software switch. Docker randomly assigns a local unoccupied private segment (defined in RFC1918) to the Docker0 interface, which is forwarded between the ports it is attached to. When creating a Docker container, a pair of Veth pair interfaces are created at the same time. This pair of interfaces is in the container, that is eth0, and the other end is local and mounted to the Docker0 bridge, with the name beginning with Veth (for example, VETHAQI2QT). In this way, the host can communicate with the container, and the container can communicate with each other.

That is, every time the container starts the IP address is not fixed, what should be done? Of course, you can write dead IP address, specify LAN network segment, to each service orchestration IP address, of course, can also put network_mode= "host", unified Host network address. Of course! These are not the best ways:

version: ' 3.7 '#服务列表Services:#基础组件 Zookeeper   Zookeeper:Image: ZookeeperRestart: AlwaysPorts:-4,181:2,181#基础组件 MySQL  DB:Image: mysql:5.7.17Command: mysqld--character-set-server=utf8mb4--collation-server=utf8mb4_unicode_ci--init-connect= ' set NAMES utf8mb4; 'Ports:-"3,636:3,306"volumes:-/var/mysqldb:/var/lib/mysql-/DOCKER/MYSQL/MY.CNF:/ETC/MYSQL/MYSQL.CONF.D/MYSQLD.CNFRestart: AlwaysEnvironment:Mysql_root_password: Password#消费者服务1 Admin  Admin:Image: "admin:2.3.1"Ports:-"7,575:8,080"depends_on:-ZookeeperRestart: AlwaysEnvironment:Zookeeper.host: zookeeper://zookeeper:2181#提供者服务1 System  system:Image: "system:2.3.1"depends_on:-Db-zookeeperRestart: AlwaysEnvironment:Zookeeper.host: zookeeper://zookeeper:2181mysql.address: db:3306

Did you see it? IP addresses are specified directly by the service name . In addition, the environment variables set in Docker can be read in Applicationcontext.xml, I am also quite surprised! ( mysql.address is configured in both the code and Docker, as set in Docker).

Then docker-compose up-d start the micro-service project will be able to ~ ~

A principle of container deployment: Try not to make changes to the file inside the container, the content to be modified is mapped to the host on a data volume, such as the MySQL configuration file and the data warehouse above.

Deploying MySQL on Docker encountered several problems, simply listed below:

1, Navicat connection time: Client does not support authentication protocol requested by server?

Solution: Go to MySQL container and run

' Root '@'%'password';

2. Expression #1 of the SELECT list is not in GROUP by clause and contains nonaggre problem?

Reason: MySQL 5.7.5 and above function depends on the detection function. If only_full_group_by SQL mode is enabled (by default), MySQL rejects the select list, and queries for the having condition or order by list refer to both unnamed non-collection columns in the GROUP by clause, and are not functionally dependent on them.

Workaround: Add the following in the MySQL configuration file:

Sql_mode=strict_trans_tables,no_zero_in_date,no_zero_date,error_for_division_by_zero,no_auto_create_user,no_ Engine_substitution

3. What is the difference between MySQL connection Parameters UseSSL=true and UseSSL=false ?

It is recommended that you do not establish an SSL connection without server authentication (an intranet environment in the same docker-compose). Depending on the MySQL 5.5.45 +,5.6.26 + and 5.7.6+ requirements, if an explicit option is not set, an SSL connection must be established by default. In order to conform to an existing application that does not use SSL. You need to explicitly disable SSL by setting UseSSL = False, or set UseSSL = True and provide a Truststore for server certificate validation .

Iv. Conclusion

Finally is a micro-service project deployment to run, almost with a minimum of docker-compose template files, so there are many places can be improved, such as MySQL password is not encrypted processing, service did not do health checks, cluster aspects have not been considered (with Docker Swarm implementation) ... etc... The road long its repair far XI, I will go up and down and quest. Share!