ETCD Enable HTTPS service

[TOC]

Cluster configuration for ETCD Direct reference to ETCD cluster deployment

This document only adds the process of SSL encryption verification based on it.

For the cluster to use SSL, you first need to generate an SSL certificate for the cluster.

We use the CFSSL series tools to generate related certificates.

Cfssl Related Tools Download

curl -s -L -o /opt/kubernetes/bin/cfssl https://pkg.cfssl.org/R1.2/cfssl_linux-amd64curl -s -L -o /opt/kubernetes/bin/cfssljson https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64chmod +x /opt/kubernetes/bin/{cfssl,cfssljson}

Generate the SSL certificate required by ETCD to generate the CA certificate

The Ca-config.json configuration is as follows:

{    "signing": {        "default": {            "expiry": "175200h"        },        "profiles": {            "kubernetes": {                "expiry": "175200h",                "usages": [                    "signing",                    "key encipherment",                    "server auth",            "client auth"                ]            },            "etcd": {                "expiry": "175200h",                "usages": [                    "signing",                    "key encipherment",                    "server auth",                    "client auth"                ]            }        }    }}

Field Description:

• Ca-config.json: You can define multiple profiles, specify different expiration times, use scenarios and so on, and then use a profile when signing a certificate. There are two profiles defined, one for kubernetes and one for ETCD. However, because this document does not involve kubernetes configuration, the Kubenretes segment is not used.
• Signing: Indicates that the certificate can be used to sign other certificates, in the generated CA.PEM certificate ca=true
• Server Auth: Indicates that the client can use the CA to authenticate the certificate provided by the server
• Client Auth: Indicates that the server can validate the certificate provided by the client with this CA

The Ca-csr.json configuration is as follows:

{    "key": {        "algo": "rsa",        "size": 2048    },    "names": [        {            "C": "CN",            "L": "Wuhan",            "ST": "Hubei",            "O": "k8s",            "OU": "System"        }    ]}

To generate a CA certificate:

cfssl gencert --initca ca-csr.json | cfssljson --bare ca

Three files are generated after execution:

• CA-KEY.PEM:CA's private key
• Ca.pem:CA Certificate
• Ca.csr:CA Certificate Request File

Generate ETCD server-side certificates

ETCD server-side certificates are used to encrypt communication between ETCD clusters

The contents of the Etcd-csr.json file are as follows:

{  "CN": "etcd-server",  "hosts": [    "localhost",    "0.0.0.0",    "127.0.0.1",    "10.5.12.16",    "10.5.12.17",    "10.5.12.18"  ],  "key": {    "algo": "rsa",    "size": 4096  },  "names": [    {      "C": "CN",      "L": "Wuhan",      "O": "k8s",      "OU": "System",      "ST": "Hubei"    }  ]}

To generate the ETCD server-side certificate:

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd

Generated three files: Etcd.pem, Etcd-key.pem, ETCD.CSR

Generate ETCD Client Certificate

ETCD client certificates are used to provide authentication methods when ETCD client connections Etcd

Etcd-client-csr.json

{  "CN": "etcd-client",  "hosts": [    ""  ],  "key": {    "algo": "rsa",    "size": 4096  },  "names": [    {      "C": "CN",      "L": "Wuhan",      "ST": "Hubei",      "O": "k8s",      "OU": "System"    }  ]}

To generate a client certificate:

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-client-csr.json | cfssljson -bare etcd-client

Generated three files: Etcd-client.pem, Etcd-client-key.pem, ETCD-CLIENT.CSR

Best practice: In real production, in order to simplify the management of ETCD, we usually do not generate a set of certificates for the server and the client, but instead generate a set of certificates that can be used for both the server and the client. When you generate a CA certificate for ETCD on top of us, you can see that there are both server auth and client auth in Etcd certificate Useags in Etcd-ca-config.json. That is, the ETCD certificate generated based on our CA certificate itself can be used for both the server and the client. However, when we generated the ETCD server certificate, we specified the hosts in the Etcd-server-csr.json, so the certificate could only be used by hosts in the specified hosts list, and all clients would be able to use this certificate. The simplest way is to leave the hosts blank just as you would when generating ETCD client certificates. In turn, that is, we can use the generated ETCD client certificate directly for the service side.

Modifying the ETCD cluster configuration file

The contents of the modified/opt/kubernetes/cfg/etcd.conf file are as follows:

ETCD_NAME=etcd1ETCD_DATA_DIR="/data/etcd"ETCD_LISTEN_CLIENT_URLS="http://10.5.12.16:2379,http://127.0.0.1:2379"ETCD_LISTEN_PEER_URLS="http://10.5.12.16:2380"ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.5.12.16:2380"ETCD_INITIAL_CLUSTER="etcd1=http://10.5.12.16:2380,etcd2=http://10.5.12.17:2380,etcd3=http://10.5.12.18:2380"ETCD_INITIAL_CLUSTER_STATE="new"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_ADVERTISE_CLIENT_URLS="http://10.5.12.16:2379"CLIENT_CERT_AUTH="true"ETCD_CA_FILE="/opt/kubernetes/ssl/ca.pem"ETCD_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"ETCD_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"PEER_CLIENT_CERT_AUTH="true"ETCD_PEER_CA_FILE="/opt/kubernetes/ssl/ca.pem"ETCD_PEER_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"ETCD_PEER_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"

Restarting the ETCD cluster

systemctl restart etcd

Verifying cluster health

When verifying the cluster health state, you need to use a client certificate to connect to the ETCD cluster:

etcdctl --cert-file=/opt/kubernetes/ssl/etcd-client.pem  --key-file=/opt/kubernetes/ssl/etcd-client-key.pem --ca-file=/opt/kubernetes/ssl/ca.pem --endpoints=https://10.5.12.16:2379,https://10.5.12.17:2379,https://10.5.12.18:2379 member list

ETCD Enable HTTPS service