Build Springboot services based on K8s, Docker, and Jenkins

Jenkins + GitHub + docker + k8s + springboot

本文介绍基于k8s、docker、jenkins、springboot构建docker服务。

Environment preparation

server-1 k8s-master Centos7 ip地址10.12.5.110server-2    k8s-node Centos7 ip地址10.12.5.115

Two services execute the following command

$ setenforce 0$ systemctl stop firewalld$ systemctl disable firewalld

Server-1 K8s-master Installation

k8s-master节点安装kubernets、docker、etcd、git、maven等软件。

Installing Docker

$ yum install docker// 修改docker配置文件 /etc/sysconfig/dockerOPTIONS=‘--registry-mirror=https://docker.mirrors.ustc.edu.cn --selinux-enabled --log-driver=journald --signature-verification=false‘ADD_REGISTRY=‘--add-registry 10.12.5.110:5000‘INSECURE_REGISTRY=‘--insecure-registry 10.12.5.110:5000‘$ systemctl enable docker$ service docker start

Installing registry

$ docker pull registry$ mkdir -p /data/docker/registry$ docker run -d -p 5000:5000 -v /data/docker/registry:/var/lib/registry registry

Installing ETCD

$ yum install etcd$ vi /etc/etcd/etcd.conf// 修改如下配置ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"$ systemctl enable etcd$ systemctl start etcd

Installing Kubernetes

$ yum install kubernetes// 修改配置文件apiserver$ vi /etc/kubernetes/apiserver// 修改如下配置KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"KUBE_API_PORT="--port=8080"KUBELET_PORT="--kubelet-port=10250"KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"// 启动服务$ systemctl enable kube-apiserver kube-controller-manager kube-scheduler$ systemctl start kube-apiserver kube-controller-manager kube-scheduler

Installing Git and MVN

$ yum install git$ yum install maven

Server-2 K8s-node Installation

k8s-master节点安装kubernets、docker等软件。

Installing Docker

$ yum install docker// 修改docker配置文件 /etc/sysconfig/dockerADD_REGISTRY=‘--add-registry 10.12.5.110:5000‘INSECURE_REGISTRY=‘--insecure-registry 10.12.5.110:5000‘$ systemctl enable docker$ systemctl start docker

Installing Kubernetes

$ yum install kubernetes// 修改配置文件kubelet$ vi /etc/kubernetes/kubelet// 修改如下配置KUBELET_ADDRESS="--address=0.0.0.0"KUBELET_PORT="--port=10250"KUBELET_HOSTNAME="--hostname-override=10.12.5.115"KUBELET_API_SERVER="--api-servers=http://10.12.5.110:8080"// 修改配置文件config$ vi /etc/kubernetes/config// 修改如下配置KUBE_MASTER="--master=http://10.12.5.110:8080"// 启动服务$ systemctl enable kube-proxy kubelet$ systemctl start kube-proxy kubelet

Check the operation of the k8s cluster

// 在k8s-master节点执行$ kubectl get nodesNAME          STATUS    AGE10.12.5.115   Ready     

In the K8s-master node builder springcloudenv Mirror

* Download jdk8$ mdkir-p/home/docker/docker-jdk8$ wget Http://javadl.oracle.com/webapps/download/AutoDL? Bundleid=233162_512cd62ec5174c3487ac17c61aaa89e8-o jre-8u161-linux-x64.tar.gz$ tar zxvf jre-8u161-linux-x64.tar.gz * Remove unused files from JDK, minimize Docker image file size $ cd jre1.8.0_171$ rm-rf lib/plugin.jar lib/ext/jfxrt.jar bin/javaws Lib/javaws.jar lib/ Desktop plugin lib/deploy* lib/*javafx* lib/*jfx* lib/amd64/libdecora_sse.so lib/amd64/libprism_*.so lib/amd64/ libfxplugins.so lib/amd64/libglass.so lib/amd64/libgstreamer-lite.so lib/amd64/libjavafx*.so lib/amd64/libjfx*.so* Write Dockerfile file $ cd/home/docker/docker-jdk8$ vi dockerfilefrom centosmaintainer by Wangtwrun cp/usr/share/zoneinfo/ Asia/shanghai/etc/localtime && Echo ' Asia/shanghai ' >/etc/timezone && yum-y install kde-l10n- Chinese && localedef-c-F UTF-8-i zh_cn zh_cn.utf8 && mkdir-p/usr/javacopy jre1.8.0_171/usr/j Ava/jre1.8.0_171env LANG ZH_CN. Utf-8env java_home/usr/java/jre1.8.0_171env PATH $JAVA _home/bin: $PATH * Create a mirrored springcloudenv and upload to registry to make sure Docker Registry has started Docker Build-t springcloudenv. $ docker Tag springcloudenv springcloudenv:v1$ Docker push springcloudenv:v1//view image in registry $ curl http://10.12.5.110:5000/v2/ _catalog{"repositories": ["Springcloudenv"]}

Installing NFS on K8s-master node installing NFS server

$ yum install -y nfs-utils rpcbind$ mkdir -p /data/mysql-pv$ chmod 777 /data/mysql-pv/$ mkdir -p /data/nfs$ chmod 666 /data/nfs/$ vi /etc/exports// 增加如下行/data/nfs 10.12.0.0/16(rw,no_root_squash,no_all_squash,sync)/data/mysql-pv 10.12.0.0/16(rw,no_root_squash,no_all_squash,sync)$ systemctl enable rpcbind nfs$ systemctl start rpcbind nfs$ mkdir -p /data/mysql$ mount -t nfs 10.12.5.110:/data/nfs /data/mysql/ -o proto=tcp -o nolock

Installing NFS on the K8s-node node

$ yum install -y nfs-utils$ mkdir -p /data/mysql$ mount -t nfs 10.12.5.110:/data/nfs /data/mysql/ -o proto=tcp -o nolock

installing MySQL Docker

以持久化方式运行myql

Creating related files on K8s-master nodes

$ mkdir-p/home/k8s/yml/services/mysql$ cd/home/k8s/yml/services/mysql$ VI mysql-pv.yamlapiversion:v1kind: Persistentvolumemetadata:name:mysql-pvspec:accessmodes:-Readwriteonce capacity:storage:1gi Persis Tentvolumereclaimpolicy:retain#storageclassname:nfs Nfs:path:/DATA/MYSQL-PV server:10.12.5.110$ VI MYSQL-PVC . Yamlapiversion:v1kind:persistentvolumeclaimmetadata:name:mysql-pvcspec:accessmodes:-ReadWriteOnce Resource S:requests:storage:1gi #storageClassName: nfs$ VI mysql.yamlapiVersion:v1kind:Servicemetadata:name:mysqls Pec:ports:-port:3306 selector:app:mysql---apiversion:extensions/v1beta1kind:deploymentmetadata:name:mysql Spec:replicas:1 Selector:matchLabels:app:mysql Template:metadata:labels:app:mysql SP Ec:containers:-Name:mysql image:mysql:5.6 env:-Name:mysql_root_password VA Lue:password Ports:- containerport:3306 name:mysql volumemounts:-Name:mysql-persistent-storage Mountpath :/var/lib/mysql volumes:-Name:mysql-persistent-storage PersistentVolumeClaim:claimName:mys Ql-pvc

Create a MySQL service and pod

$ cd /home/k8s/yml/services/mysql$ kubectl create -f mysq-pv.yaml$ kubectl get pvNAME       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM               REASON    AGEmysql-pv   1Gi        RWO           Retain          Bound     default/mysql-pvc             2d$ kubectl create -f mysq-pvc.yaml$ kubectl get pvcNAME        STATUS    VOLUME     CAPACITY   ACCESSMODES   AGEmysql-pvc   Bound     mysql-pv   1Gi        RWO           2d$ kubectl create -f mysq.yaml$ kubectl get serviceNAME         CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGEmysql        10.254.151.113   <none>        3306/TCP   2d$ kubectl get podNAME                           READY     STATUS             RESTARTS   AGEmysql-3827607452-hd5ct         1/1       Running            80         19h

K8s-master node installation and configuration Jenkins

$ mkdir /home/jenkins$ cd$ vi .bash_profileexport JENKINS_HOME=/home/jenkins$ cd /home/jenkins$ wget http://mirrors.jenkins.io/war-stable/latest/jenkins.war$ java -jar jenkins.war --httpPort=9090&浏览器打开地址：http://10.12.5.110:9090把文件/home/jenkins/secrets/initialAdminPassword的内容复制到安装页面解决插件安装离线问题打开地址：http://10.12.5.110:9090/pluginManager/advanced把https://updates.jenkins.io/update-center.json修改为http://updates.jenkins.io/update-center.json安装插件：Publish Over SSH

Create a Jenkins task with source code on GitHub

* Task Name: springboottest* task type: Free Style * Create github access credentialcredential type: Username with password* source management Select Gitrepository URL A value of https://github.com/wangtiewu/springboottest.gitCredentials creates a build script for the credential* created in the previous step mvn clean packagetag=$ (date + '%y%m%d%h%m ') App=springcloudtestecho $tag > Version_$app.txtdocker_name= $appdocker build-t $docker _name: $tag. Docker push $docker _name: $tag * Create SSH Server System Management-〉 system settings menu-〉publish over ssh* add post build operation, Send build artifacts over Sshssh Server   : The name value is k8s mastertransfer set:source files value is version_springcloudtest.txttransfer set:remote directory The value is springcloudtesttransfer set:exec the command value is Cd/home/k8s/yml/services/springcloudtestapp=springcloudtestdeploy_ file= $app. yamlversion_file=version_$app.txtpatch_file= $app. patchnew_version=$ (Cat $version _file) old_version=$ ( Cat $deploy _file | grep image | Awk-f ":" ' {print $} ') sed-i "s/$old _version/$new _version/" $deploy _filedeployment_num=$ (/usr/bin/kubectl get Deploym ent | grep springcloudtest | WC-L) If [$deplOyment_num-eq 1];then p_old_version=$ (cat $patch _file | jq '. Spec.template.spec.containers[0].image ') p_new_vers Ion=\ "$app: $new _version\" Sed-i "s/$p _old_version/$p _new_version/" $patch _file patch=$ (cat $patch _file)/usr/b In/kubectl Patch Deployment $app-P $patchelse/usr/bin/kubectl create-f $deploy _filefi

Create Springclouttest service and pod

$ mkdir -p /home/k8s/yml/services/springcloudtest$ cd /home/k8s/yml/services/springcloudtest$ vi springcloudtest-svr.yamlapiVersion: v1kind: Servicemetadata:  name: springcloudtestspec:  ports:  - name: springcloudtest-svr    port: 9091    nodePort: 30000    targetPort: 9091  selector:    app: springcloudtest  type: NodePort$ vi springcloudtest.yamlapiVersion: extensions/v1beta1kind: Deploymentmetadata:  name: springcloudtestspec:  replicas: 1  template:    metadata:      labels:        app: springcloudtest    spec:      containers:      - name: springclouttest        image: springcloudtest:201805272108        ports:         - containerPort: 9091          protocol: TCP$ vi springcloudtest.patch{"spec":{"template":{"spec":{"containers":[{"name":"springclouttest","image":"springcloudtest:201805052039"}]}}}}

Build Jenkins Task: Springclouttest

构建完成后，kubectl get pod 查看springcloudtest-1053936621-w19sh   1/1       Running   1          1d

Create Springclouttest Service

$ cd/home/k8s/yml/services/springcloudtest$ kubectl create-f springcloudtest-svr.yaml View service status $ kubectl get        Servicespringcloudtest 10.254.126.13 <nodes> 9091:30000/tcp 45m in K8s-node view container case $ docker Pscontainer ID              IMAGE COMMAND CREATED STATUS PORTS names269366fa1b49 mysql:5.6 "Docker   -entrypoint ... " About a hour ago up about an hour k8s_mysql.affc4af4_mysql-3827607452-lp1cp_default_15ace557-60e  d-11e8-aa6f-0800276c68b7_404d6f6a2ff21fb61984 springcloudtest:201805272108 "/bin/sh   -C ' Java ... ' About a hour ago up about an hour k8s_springclouttest.c0ee0b5d_springcloudtest-1053936621-w19sh_ DEFAULT_A320F883-61B2-11E8-AA6F-0800276C68B7_A99E1B3788E9BA47FAF1 registry.access.redhat.com/rhel7/ Pod-infrastructure:lAtest "/usr/bin/pod" about a hour ago up about an hour k8s_pod.389b0ddb_mysql-382760 7452-LP1CP_DEFAULT_15ACE557-60ED-11E8-AA6F-0800276C68B7_7387FDC6130A60ACD94D registry.access.redhat.com/rhel7/ Pod-infrastructure:latest "/usr/bin/pod" about a hour ago up about an hour K8s_pod.2 aaa0bac_springcloudtest-1053936621-w19sh_default_a320f883-61b2-11e8-aa6f-0800276c68b7_fa8aba88 solve the Nodeport mode, Node's node can access the service, and the client request service will fail at the same point. $ iptables-p FORWARD ACCEPT

Creating databases and Tables

Testing Services

浏览器打开 http://10.12.5.115:30000/浏览器显示 Hello World!$ curl --data ‘{"id":1,"name":"Hello World"}‘ -H "Content-type: Application/json" http://10.12.5.115:30000/create/hello

Build Springboot services based on K8s, Docker, and Jenkins