Use Traefik as the ingress of Kubernetes

[TOC]

Description

There is no detailed explanation of the Kubernetes's service exposure and Traefik's principles. The Traefik principle can be consulted in the official documentation: https://docs.traefik.io/, This document can also be consulted about the service exposure of kubernetes and the rationale for using Traefik as a kubernetes ingress: https://mritd.me/2016/12/06/ Try-traefik-on-kubernetes/. This document focuses on the actual operation of Traefik as a kubernetes ingress, including enabling HTTPS in Traefik, outputting Traefik logs as JSON, and collecting them through filebeat. and monitoring Traefik through Prometheus.

Deployment creates a separate namespace

kubectl create ns ingress

Configure RBAC Authorization

I use 1.8 here Kubernetes and RBAC authorization is enabled. The official Traefik-rbac.yaml file can be consulted: https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/ Traefik-rbac.yaml, but in my actual test, there is still a problem. I have directly adopted a lazy approach here, directly to the maximum permissions. In the following configuration file, a serviceaccount name called Traefik-ingress-controller is created, and the Cluster-admin permissions are granted directly to this ServiceAccount

The Traefik.rbac.yaml example is as follows:

apiVersion: v1kind: ServiceAccountmetadata:  labels:    k8s-app: traefik-ingress-controller  name: traefik-ingress-controller  namespace: ingress---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:  name: traefik-ingress-controller  labels:    k8s-app: traefik-ingress-controllerroleRef:  apiGroup: rbac.authorization.k8s.io  kind: ClusterRole  name: cluster-adminsubjects:- kind: ServiceAccount  name: traefik-ingress-controller  namespace: ingress

Configure Secret

When we use Traefik as ingress, we need to process the HTTPS request at the same time, so we need to configure the certificate file for Traefik. Here the certificate file and the key file are put into secret. The names of my two files here are DZ11.COM.CRT and Dz11.com.key respectively. The secret method is created as follows:

kubectl create secret tls dz11-ingress-secret --key dz11.com.key --cert dz11.com.crt -n ingress

Create a configmap to hold the Traefik configuration file

The Traefik.configmap.yaml configuration is as follows:

apiVersion: v1kind: ConfigMapmetadata:  name: traefik-conf  namespace: ingressdata:  traefik.toml: |    defaultEntryPoints = ["http", "https"]    [entryPoints]      [entryPoints.http]      address = ":80"      [entryPoints.https]      address = ":443"        [entryPoints.https.tls]          [[entryPoints.https.tls.certificates]]          certFile = "/keys/dz11.com.crt"          keyFile = "/keys/dz11.com.key"    #traefikLogsFile = "log/traefik.log"    [accessLog]    filePath = "/logs/traefik.access.log"    format = "json"

It should be noted that I am here to support both HTTP and HTTPS, and do not do HTTP force to jump to HTTPS. If you need HTTP forcing to jump to HTTPS, you can refer to the following configuration:

defaultEntryPoints = ["http", "https"][entryPoints]  [entryPoints.http]  address = ":80"    [entryPoints.http.redirect]    entryPoint = "https"  [entryPoints.https]  address = ":443"    [entryPoints.https.tls]      [[entryPoints.https.tls.certificates]]      certFile = "/keys/dz11.com.crt"      keyFile = "/keys/dz11.com.key"#traefikLogsFile = "log/traefik.log"[accessLog]filePath = "/logs/traefik.access.log"format = "json"

In addition, in the above configuration, I started the Traefik access log and specified the format as JSON, which was designed to facilitate the collection of filebeat directly from the back. Here also need to explain, Traefik two kinds of logs, one is the log of the Traefik service itself, and the other is the access log. I only have access logs enabled here. The default Traefik service log is printed through the container's standard output.

Configuring the Traefik Deployment file

Here I use the Nodeselecter method to fix the Traefik on the specified two nodes, if deployed in deployment way, you need to label the selected nodes first.
The Traefik.dm.yaml configuration is as follows:

APIVERSION:EXTENSIONS/V1BETA1KIND:DEPLOYMENTMETADATA:NAME:TRAEFIK-INGRESS-LB namespace:ingress Labels:k8s-app: Traefik-ingress-lbspec:strategy:type:rollingupdate rollingupdate:maxunavailable:1 maxsurge:0 Repl Icas:2 selector:matchlabels:k8s-app:traefik-ingress-lb Template:metadata:labels:k8s-app: TRAEFIK-INGRESS-LB annotations:prometheus.io/scrape: "True" Prometheus.io/port: "8580" spec:t Erminationgraceperiodseconds:60 hostnetwork:true restartpolicy:always Volumes:-name:traefik-conf IG Configmap:name:traefik-conf-name:traefik-key secret:secretname:dz11-ingress          -secret-name:traefik-log Hostpath:path:/mnt/srvlogs-name:localtime hostpath: Path:/etc/localtime containers:-image:dyhub.douyucdn.cn/library/traefik:v1.4.3 name:traefik-ing ress-lb# Resources:# limits:# cpu:200m# memory:30mi# requests:# cpu:100m# Memory:20mi securityContext:privileged:true Ports:-Name:http Containerport:8        0-name:https containerport:443-name:admin containerport:8580 volumemounts:        -Mountpath: "/config" Name: "Traefik-config"-Mountpath: "/logs" Name: "Traefik-log"        -Mountpath: "/keys" Name: "Traefik-key"-Mountpath: "/etc/localtime" Name: "LocalTime"        Args:---configfile=/config/traefik.toml---web---web.address=:8580---kubernetes        ---web.metrics.prometheus Serviceaccountname:traefik-ingress-controller Nodeselector:proxy: "true" Ingress: "Traefik"

It is necessary to note that the access log and the HTTPS certificate and key are mounted in a volumemounts manner. Then in the startup parameter, specify the--web.metrics.prometheus parameter to expose metrics for Prometheus Collection, and specify the management port to 8580.

Finally, create all the resources configured above:

kubectl create -f ./

To this, Traefik is deployed on kubernetes.

Configure the Service

Here is an example of Traefik-ui as the proxy backend for Traefik.

An example of creating a service file first Traefik-ui.svc.yaml is as follows:

apiVersion: v1kind: Servicemetadata:  name: traefik-web-ui  namespace: ingressspec:  clusterIP: None  selector:    k8s-app: traefik-ingress-lb  ports:  - name: web    port: 8580    targetPort: 8580

Then create a Traefik-ui.ingress.yaml file as follows:

apiVersion: extensions/v1beta1kind: Ingressmetadata:  annotations:    kubernetes.io/ingress.class: traefik  name: traefik-web-ui  namespace: ingressspec:  tls:#  - hosts:#    - traefik-ui.dz11.com  - secretName: dz11-ingress-secret  rules:  - host: traefik-ui.dz11.com    http:      paths:      - path: /        backend:          serviceName: traefik-web-ui          servicePort: web

To create a related resource:

kubectl create -f ./traefik-ui.svc.yamlkubectl create -f ./traefik-ui.ingress.yaml

Configure DNS resolution to access the Traefik-ui service through traefik-ui.dz11.com, HTTP and HTTPS support at the same time, and do not force jumps.

Monitor Traefik with Prometheus

When you start Traefik, the--web.metrics.prometheus option is used, only the IP and admin ports of the Traefik service are added to the Prometheus configuration file. As follows:

  - job_name: ‘traefik‘    static_configs:    - targets: [‘10.1.61.147:8580‘,‘10.1.61.138:8580‘]

Restart Prometheus:

systemctl restart prometheus

In this example, Prometheus is not running in Kubernetes, but is deployed independently

In the Prometheus's status target, view:

Add Dashboard in Grafana, where the official ID is 2870 dashboard, added after the effect is as follows:

Use Traefik as the ingress of Kubernetes