Distributed Service Coordinator Zookeeper-application scenarios and monitoring

Zookeeper is the role of coordinator in Distributed system, which can be applied to the implementation of leader election, distributed lock, configuration management and other services. Below we learn and understand zookeeper (ZK) from the three aspects of API, application scenarios and monitoring provided by zookeeper.

ZK API

ZK manages stored data in the form of a UNIX file system tree structure, as shown below:

Each of these tree nodes is called Znode, and each znode resembles a file that contains file meta information (meta data) and data.

The following we use the server to express the ZK service provider, theclient represents the user of the ZK service, when the client connects ZK, the corresponding session information is created.

There are two types of Znode:

Regular: The type Znode can only be explicitly created or deleted by the client side

Ephemeral: Client can create or delete this type of Znode; When the session terminates, ZK also deletes the type Znode

Znode can also be played on the sequential flag, the flag is played on the Znode, will add their own digital suffix

ZK provides the following APIs for the client to manipulate the data stored in Znode and Znode:

• Create (path, data, flags): Creates a path of Znode, where the data[] data is stored, flags can be set to regular or ephemeral, and optionally marked with a sequential flag.
• Delete (path, version): Delete the znode of the corresponding path/version
• Exists (Path,watch): Returns True if path corresponding to Znode is present, otherwise the FALSE,WATCH flag can be set to listen for events
• GetData (Path, watch): Returns data and meta information for the corresponding znode (such as version, etc.)
• SetData (path, data, version): writes data[] data to the znode of the corresponding path/version
• GetChildren (Path, watch): Returns the collection of child nodes for the specified Znode

ZK Application Scenario

Based on the operation of Znode and Znode data provided by ZK, it is easy to implement leader election, distributed lock, configuration management and other services.

Leader elections

Using the sequential logo ephemeral, we can achieve the leader election. Suppose you need to select leader from three clients, the implementation process is as follows:

1, each create ephemeral type of znode, and play on the sequential logo:

[Zk:localhost:21814ls /master[lock-0000000241, lock-0000000243 , lock-0000000242]

2, check all the Znode under the/master path, if you create the Znode sequence number is the smallest, you think you are leader; otherwise the record sequence number is smaller than the Znode

3, non-leader on the secondary small serial number Znode set the Listener event, and repeat the above steps 2

If the above/master/lock-0000000241 node is deleted (the corresponding client service exception or network exception, etc.), then/master/lock-0000000242 corresponding Znode will promote themselves to leader. The client only cares about the Znode and the minor znode, which avoids the surprise group effect (herd Effect).

The implementation of the distributed lock is the same as the implementation of the above leader elections, and we can also implement the lease mechanism (time-limited authorization service) based on ZK.

Configuration Management

Znode can store data, based on this, we can use ZK to implement distributed system configuration management, assuming that there is a service A,a expansion device needs to synchronize the corresponding new Ip/port to the full network server a.conf configuration, the implementation process is as follows:

1, a expansion, the corresponding in ZK added Znode, the Znode data form as follows:

 [Zk:localhost:2181  (CONNECTED) 30  ] Get/a/blk-0000340369   " svr_info  " : [{ " ip  " :  1.1.1.1.  , "  port  : "   11000   }]}czxid  = 0x2ffdeda3be   

2, the whole network machine monitoring/A, when the Znode under the new node join, call the corresponding processing function, the new Ip/port service A to join a.conf

3. After completing step 2, continue to set the monitor for/a

The steps for service scaling are similar, when the machine is offline, the ZK node is deleted, and the whole network machine listens to the event and rejects the device in the configuration.

ZK Monitoring

ZK itself provides some "four word command", through these four-word command, we can get the ZK cluster, the role of a ZK, the number of Znode, health status and other information:

#Echo "mntr"| /usr/bin/netcat127.0.0.1 2181zk_version3.4.3-1240972, built on Geneva/ ./ - Ten: -gmtzk_packets_received267044485zk_packets_sent267069992zk_outstanding_requests0 zk_server_state followerzk_znode_count16216

The commonly used four-word commands are:

• mntr: Displays information such as its role, number of znode, average call time, number of packets received, etc.
• Ruok: Diagnosing whether the state itself is OK
• cons: Displaying the current client connection

Like not to ask a drunken person whether or not drunk, we are not sure that a reply to "Imok" ZK is really OK, we can create/delete Znode through ZK's own zkcli.sh simulation client:

/usr/local/zookeeper/bin/zkcli. SH ' Test ' >/dev/null2>&1/usr/local/zookeeper/bin/zkcli. sh delete/zookeeper/test >/dev/null2>&1

According to the return value to determine whether to add, delete Znode is successful, so as to determine whether the state of ZK is normal.

Summary

Zookeeper manages data in the form of a directory tree, provides interfaces such as Znode monitoring, data setup, and, based on these interfaces, we can implement functions such as leader elections, configuration management, naming services, and so on. With the four-word command combined with the Zookeeper client creation/deletion Znode, we can implement effective monitoring of the zookeeper. In a variety of distributed systems, we can often see zookeeper figure.

Reference:zookeeper:wait-free Coordination for Internet-scale Systems

Distributed Service Coordinator Zookeeper-application scenarios and monitoring