[Experience Exchange] active-active Mode setting kubernetes Master Multi-node high availability

About Kubernetes Master Multi-node and high-availability, the online approach takes a active-standby approach, namely:

Using software such as pacemaker makes certain master services (Apiserver,scheduler,controller-manager) run only one instance at a time. Specifically, if you have more than one master node, Scheduler,controller-manager is installed on it, Apiserver:

• For Schduler services, run only on one master node at a time,
• For Controller-manager, run only on one master node at a time,
• For Apiserver, run only on one master node at a time,

The reliability of this method needs to be explored, at least I have had this problem: once the pacemaker problem or the administrator mistakenly operation, on multiple nodes at the same time to start the scheduler or Controller-manager, will cause system failure.

In fact, refer to the Kubernetes latest documentation, the official has relevant tips:

Master Elected components

So far we had set up state storage, and we had set up the API server, but we haven ' t run anything that actually Modi Fies cluster state, such as the Controller manager and scheduler. To achieve this reliably, we are want to having one actor modifying state at a time, but we want replicated instances of th ESE actors, in case a machine dies. To achieve this, we is going to use a lease-lock with the API to perform master election. We'll use --leader-elect the flags for each scheduler and Controller-manager, using a lease in the API would ensure that's only 1 ins Tance of the Scheduler and Controller-manager is running at once.

That is, Controller-manager and scheduler as long as the--leader-elect=true parameters can be started at the same time, the system will automatically elect leader. Apiserver would have been able to run multiple nodes at the same time, as long as they were connected to the same ETCD cluster.

Following the above ideas, the Master HA architecture can be designed as follows:

Note:

1. First set ETCD cluster

About ETCD's cluster setting method, has the mature document on the net to be able to refer;

2. The apiserver of each master node starts simultaneously, pointing to the same ETCD cluster

3. Apiserver Service Set VIP

It is recommended to support load balancing through KEEPALIVED+LVS or haproxy settings;

4. Controller-manager and Scheduler plus--leader-elect=true parameters of each master node are started simultaneously

Through the above settings, all the related services on the master node are active, and if one node or one service hangs, it does not affect the entire cluster's operation.

[Experience Exchange] active-active Mode setting kubernetes Master Multi-node high availability