Distributed Service Coordinator Zookeeper-application scenarios and monitoring

Zookeeper is the role of coordinator in Distributed system, which can be applied to the implementation of leader election, distributed lock, configuration management and other services. Below we learn and understand zookeeper (ZK) from the three aspects of API, application scenarios and monitoring provided by zookeeper.

ZK API

ZK manages stored data in the form of a UNIX file system tree structure, as shown below:

Each of these tree nodes is called Znode, and each znode resembles a file that contains file meta information (meta data) and data.

The following we use the server to express the ZK service provider, theclient represents the user of the ZK service, when the client connects ZK, the corresponding session information is created.

There are two types of Znode:

Regular: The type Znode can only be explicitly created or deleted by the client side

Ephemeral: Client can create or delete this type of Znode; When the session terminates, ZK also deletes the type Znode

Znode can also be played on the sequential flag, the flag is played on the Znode, will add their own digital suffix

ZK provides the following APIs for the client to manipulate data stored in Znode and Znode: Create (path, data, flags): Creates a znode with path, where data[] data is stored, Flags can be set to regular or ephemeral, and the sequential flag can be selected. Delete (path, version): Deletes the Znode exists (path,watch) of the corresponding path/version: Returns True if there is a path corresponding to Znode, otherwise returns the FALSE,WATCH flag to set the listener event GetData (Path, watch): Returns data and meta information corresponding to Znode (such as version, etc.) SetData (path, data, version): writes data[] data to Path/version of the corresponding Znode GetChildren (Path, watch): Returns the collection of child nodes for the specified Znode

ZK Application Scenario

Based on the operation of Znode and Znode data provided by ZK, it is easy to implement leader election, distributed lock, configuration management and other services.

leader Elections

Using the sequential logo ephemeral, we can achieve the leader election. Suppose you need to select leader from three clients, the implementation process is as follows:

1, each create ephemeral type of znode, and play on the sequential logo:

[Zk:localhost:2181 (CONNECTED) 4] ls/master
[lock-0000000241, lock-0000000243, lock-0000000242]

2, check all the Znode under the/master path, if you create the Znode sequence number is the smallest, you think you are leader; otherwise the record sequence number is smaller than the Znode

3, non-leader on the secondary small serial number Znode set the Listener event, and repeat the above steps 2

If the above/master/lock-0000000241 node is deleted (the corresponding client service exception or network exception, etc.), then/master/lock-0000000242 corresponding Znode will promote themselves to leader. The client only cares about the Znode and the minor znode, which avoids the surprise group effect (herd Effect).

The implementation of the distributed lock is the same as the implementation of the above leader elections, and we can also implement the lease mechanism (time-limited authorization service) based on ZK.

Configuration Management

Znode can store data, based on this, we can use ZK to implement distributed system configuration management, assuming that there is a service A,a expansion device needs to synchronize the corresponding new Ip/port to the full network server a.conf configuration, the implementation process is as follows:

1, a expansion, the corresponding in ZK added Znode, the Znode data form as follows:

[Zk:localhost:2181 (CONNECTED)] get/a/blk-0000340369
{"Svr_info": [{"IP": "1.1.1.1.", "Port": "11000"}]}
Czxid = 0x2ffdeda3be ...

2, the whole network machine monitoring/A, when the Znode under the new node join, call the corresponding processing function, the new Ip/port service A to join a.conf

3. After completing step 2, continue to set the monitor for/a

The steps for service scaling are similar, when the machine is offline, the ZK node is deleted, and the whole network machine listens to the event and rejects the device in the configuration.

ZK Monitoring

ZK itself provides some "four word command", through these four-word command, we can get the ZK cluster, the role of a ZK, the number of Znode, health status and other information:

# echo "Mntr" | /usr/bin/netcat 127.0.0.1 2181
zk_version      3.4.3-1240972, built on 02/06/2012 10:48 GMT
zk_packets_ Received     267044485
zk_packets_sent 267069992
zk_outstanding_requests 0
zk_server_state Follower
Zk_znode_count  16216

The commonly used four-word command is: mntr: Shows the characters, Znode number, average call time, the number of packets received, etc. Ruok: Diagnosing the status of itself OK cons: Show current Client connection

Like not to ask a drunken person whether or not drunk, we are not sure that a reply to "Imok" ZK is really OK, we can create/delete Znode through ZK's own zkcli.sh simulation client:

/usr/local/zookeeper/bin/zkcli.sh create/zookeeper/test ' test ' >/dev/null 2>&1
/usr/local/zookeeper/ bin/zkcli.sh delete/zookeeper/test >/dev/null 2>&1

According to the return value to determine whether to add, delete Znode is successful, so as to determine whether the state of ZK is normal.

Summary

Zookeeper manages data in the form of a directory tree, provides interfaces such as Znode monitoring, data setup, and, based on these interfaces, we can implement functions such as leader elections, configuration management, naming services, and so on. With the four-word command combined with the Zookeeper client creation/deletion Znode, we can implement effective monitoring of the zookeeper. In a variety of distributed systems, we can often see zookeeper figure.

Reference:zookeeper:wait-free Coordination for Internet-scale Systems