Zookeeper core mechanism

Zookeeper is a sub-project under Hadoop that is a reliable coordination system for large distributed systems, providing features such as naming services, configuration maintenance, distributed synchronization, cluster services, and more.

　Zookeeper are cluster-replicated, and the Zab (Zookeeper Atomic Broadcast) protocol is used to maintain data consistency across clusters.

the agreement includes2Stages of:leader electionStages andactomic Broadcastphase. The cluster will elect aleader, the other machines are calledFollower, all write operations are sent toleader, and throughBroadcasttell all the updates toFollower. Whenleadercrashes orleaderlose most ofFollowerThe need to re-elect a newleader, so that all servers are restored to a correct state. Whenleaderare elected, and most servers have completed andleaderwhen the state is synchronized,leader electionthe process is over and will enterAtomic Broadcastthe process. actomic BroadcastSyncleaderand theFollowerbetween the information to ensureleaderand theFollowerhave the same system state。

the structure diagram of the Zookeeper cluster is as follows:

Implementation of Routing and load Balancing

　　As services become more and more large, the number of corresponding machines is becoming larger, and it is becoming more and more difficult to manage and maintain the configuration information of services and addresses by manual. And, relying on a single hardware load balancing device or using LVS,Nginx and other software schemes for routing and load-balancing scheduling, the problem of single point of failure is also beginning to highlight, once the service routing or load balancing server outage, All services that depend on them will be invalidated. With a dual-machine, highly available deployment scenario, using a single server "standby" can partially solve the problem, but given the high cost of load balancing devices, it is difficult to fully promote it.

　　 Once the server andZooKeeperThe cluster is disconnected and the node does not exist, by registering the correspondingWatcher, the service consumer is able to be informed of changes to the service provider's machine information in the first time. Using itsZnodethe characteristics andWatchermechanism as a configuration center for dynamic registration and access to service information, unified Management service name and its corresponding server list information, we can perceive the status of back-end server in near real-time(On- line, offline, downtime). Zookeeperinter-cluster throughZabprotocol, the service configuration information remains consistent andZookeeperits own fault-tolerant features andleaderthe electoral mechanism ensures that we can easily expand our capacity.

Zookeeper znode non-persistent features, The corresponding machine can be removed dynamically from the service configuration center and trigger the service consumer's watcher machine online or offline watcher re-querying the service address. This kind of unstructured structure, so that service consumers in the service information is not changed, almost do not rely on the configuration center, to solve the previous load balancing equipment caused by the single point of failure, and greatly reduce the pressure of the service configuration center.

by Zookeeper to realize dynamic awareness of service dynamic registration, on-line and offline, the expansion is convenient, the fault tolerance is good, and the non-centralized structure can solve the single point of failure problem which was brought by the load balancing equipment before. The service consumer will only be able to get the latest service address list on Zookeeper when the configuration information is updated, and the local cache can be used at other times, so that the service consumer does not rely on the configuration center when the service information is not changed. Can greatly reduce the pressure of the configuration center.

Zookeeper core mechanism