ZooKeeperNote-Common Operations and Applications

ZooKeeper is a distributed open-source Coordination Service for distributed applications. It uses a set of simple operation primitives to enable distributed applications to implement high-level services, such as synchronization, configuration maintenance, group and naming management. ZK features high performance, high availability (replication), and ordering. See the previous article: zookeeper

ZooKeeper is a distributed open-source Coordination Service for distributed applications. It uses a set of simple operation primitives to enable distributed applications to implement high-level services, such as synchronization, configuration maintenance, group and naming management. ZK features high performance, high availability (replication), and ordering. See the previous article: zookeeper

ZooKeeper is a distributed open-source Coordination Service for distributed applications. It uses a set of simple operation primitives to enable distributed applications to implement high-level services, such as synchronization, configuration maintenance, group and naming management. ZK features high performance, high availability (replication), and ordering. See the previous article. zookeeper: A Distributed Coordination Service for distributed applications. This document briefly introduces the frequently used methods in Development (ZooKeeper API ).

1. create

Class:Org. apache. zookeeper. ZooKeeper
public String create(String path, byte[] data, List acl, CreateMode createMode) throws KeeperException, InterruptedException
public void create(String path, byte[] data, List acl, CreateMode createMode, AsyncCallback.StringCallback cb, Object ctx)
Create a node with a given path and set the data and access control list (acl) for it ). The node in ZooKeeper has a directory structure that is "directory" and "regular file" relative to the directory structure in the file system ". The second create method is the asynchronous version of create. When the creation is complete, asynchronous callback is called.

(1). org. apache. zookeeper. data. ACL

This part is excerpted to explain the ACL in Zookeeper.
ZooKeeper uses ACL to control access to ZNode. The ZooKeeper client specifies the ACL list for znode. The ZooKeeper server determines whether the client requesting ZNode has the corresponding operation permission based on the ACL list.
An ACL object consists of schema: ID and Permissions.
A). scheme: Scheme corresponds to the scheme used for permission management. zookeeper implements a ACL Gable ACL scheme. You can extend the ACL mechanism by extending scheme. Zookeeper-3.4.4 by default supports the following scheme:
World: there is only one id named anyone, and world: anyone represents anyone. zookeeper has permissions on all nodes that belong to world: anyone.
Auth: it does not require an id, as long as all users who use authentication have permissions (zookeeper supports authencation through kerberos, and also supports authentication in the form of username/password)
Digest: The corresponding id is username: BASE64 (SHA1 (password). It needs to authentication in the form of username: password first.
Ip Address: The corresponding id is the ip address of the client. You can set an ip segment, for example, IP Address: 192.168.1.0/16, to match the ip segment of the first 16 bits.
Super: In this scheme scenario, the corresponding id has super permissions and can do anything (cdrwa)
B). perm. ZooKeeper has five Permissions: READ, WRITE, CREATE, DELETE, and ADMIN from low to high. ACL Permissions can be one or more of the five Permissions, their meanings are:
* READ: allows you to obtain the value of a node and list its subnodes.
* WRITE: allows you to set the value of this node.
* CREATE: subnodes can be created.
* DELETE: You can DELETE subnodes.
* ADMIN: allows you to set permissions for this node.

(2). org. apache. zookeeper. CreateMode

Org. apache. zookeeper. CreateMode can be used to set whether znode is EPHEMERAL or SEQUENTIAL. It can be the following four values:
PERSISTENT persistence directory znode
PERSISTENT_SEQUENTIAL: Directory znode that is automatically numbered sequentially. This directory node increases progressively according to the number of existing nodes.
The temporary directory znode of EPHEMERAL. Once the client and server of the znode are disconnected, the node is automatically deleted. Temporary nodes (EPHEMERAL) cannot have child node data
EPHEMERAL_SEQUENTIAL is automatically numbered znode.

(3). zkCli command

The create Command encapsulation is implemented in zkCli, allowing users to test and manage data:
Create [-s] [-e] path data acl
"-S" indicates creating a node with automatic serial numbers and "-e" indicates creating a temporary node. The default value is persistent node.
For example:
Create a permanent node and a temporary Node

create  /test nullCreated /testcreate -e /test0 nullCreated /test0

Create a node with automatic sequential numbers. The ACL is "digest" (username: test password: debugo) and the ACL is "all" (rwcda ). For the generation of digest, see DigestAuthenticationProvider in zookeeper. generateDigest (String ipName) method. You can specify the original username and password to obtain the String after "digest", for example, pass "test: test ", test: V28q/NynI4JI3Rk54h0r8O5kMug = "is obtained. The internal principle is to perform MD5 + sha1 operations on the" password "part.

create -s /test0/test null digest:test:V28q/NynI4JI3Rk54h0r8O5kMug=:rwcdaEphemerals cannot have children: /test0/testcreate -s /test/test null digest:test:V28q/NynI4JI3Rk54h0r8O5kMug=:rwcdaCreated /test/test0000000000

Create a node. Its ACL uses ip address (172.19.20./ 24) and only has read permission.

create /test/test1 "hello world" ip:172.19.17.0/24:rCreated /test/test1

2. exist

Class:Org. apache. zookeeper. ZooKeeper
public Stat exists(String path, Watcher watcher) throws KeeperException, InterruptedException
Returns whether the znode of a path exists. And set whether to monitor this node (the second parameter is boolean watcher ). When the second parameter is true and the statement is successfully executed, the listener (watcher) is triggered when the node is successfully created, deleted, or modified.
exists(String, Watcher)
Overload method. Here you can specify a specific listener (watcher) object.
exists(String, Watcher, AsyncCallback.StatCallback, Object)
exists(String, boolean, AsyncCallback.StatCallback, Object)
Asynchronous Implementation of exist

3. delete

public void delete(String path, int version) throws InterruptedException, KeeperException
Delete znode corresponding to path. If version is-1, all data of this node can be deleted. In addition, delete also has an asynchronous version.
delete(String path, int version, AsyncCallback.VoidCallback cb, Object ctx)
Asynchronous version of delete.
For example, remove a node from zkCli:

delete /test/test1

4. getChildren

public List getChildren(String path, boolean watch) throws KeeperException, InterruptedException
Obtain all znodes in the specified path. This method is the same as exist. You can set watcher/to specify a specific Watcher object.

5. setData & getData

Stat setData(String path, byte[] data, int version)
When a node with a given path exists, set the data for the path. You can specify the version number of the data. If version is-1, any version can be matched.
void setData(String path, byte[] data, int version, AsyncCallback.StatCallback cb, Object ctx)
The asynchronous version of setData.
byte[] getData(String path, Watcher watcher, Stat stat)
Obtain the data of the znode node corresponding to this path. The data version and other information can be specified through stat.
void getData(String path, Watcher watcher, AsyncCallback.DataCallback cb, Object ctx)
The asynchronous version of getData.

6. setACL and getACL

Stat setACL(String path, List acl, int version)
To reset the access permission for a znode node, note that the directory node permissions in ZooKeeper do not have the transmission permission. the permissions of the parent znode node cannot be passed to the sub-directory node. The ACL setting method has been introduced in create. You can set a series of ACL rules (that is, specifying a series of ACL objects ).
void setACL(String path, List acl, int version, AsyncCallback.StatCallback cb, Object ctx)
Asynchronous version of setACL
List getACL(String path, Stat stat)
Returns the list of ACL objects of a znode node.
void getACL(String path, Stat stat, AsyncCallback.ACLCallback cb, Object ctx)
Asynchronous version of getACL
For example, set an ACL rule in zkCli:

[zk: localhost:2181(CONNECTED) 43] setAcl /test world:anyone:rcZxid = 0xf000500edctime = Wed Sep 24 15:13:29 CST 2014......[zk: localhost:2181(CONNECTED) 44] getAcl /test'world,'anyone: r

The following content is from the distributed service framework Zookeeper-manage data in the distributed environment, which is well written. It is extracted for reference only ^.
Zookeeper is a distributed service management framework designed based on the observer mode. It stores and manages data that everyone cares about, and then accepts the registration of the observer, once the status of the data changes, Zookeeper will be responsible for notifying the observer who has registered on Zookeeper to respond accordingly, so as to implement the cluster management mode similar to Master/Slave.
ZooKeeper has the following applications:
NameService)
ConfigurationManagement)
GroupMembership)
Locks)
Queue Management

1. Unified Naming Service (NameService)

In distributed applications, a complete set of naming rules is usually required, which can generate unique names and make it easy for people to recognize and remember. Generally, a tree name structure is an ideal choice, the tree name structure is a hierarchical directory structure, which is user-friendly and does not duplicate. Speaking of this, you may have thought of JNDI. Yes, the Name Service of Zookeeper is similar to that of JNDI. They all associate hierarchical directory structures with certain resources, however, the Zookeeper NameService is more broadly associated. You may not need to associate a name with a specific resource. You may only need one name that will not be repeated, just like a database that generates a unique digital primary key.
NameService is already a built-in function of Zookeeper. You only need to call the Zookeeper API. If you call the create interface, you can easily create a directory node.

2. ConfigurationManagement)

Configuration Management is very common in distributed application environments. For example, the same application system requires multiple pcservers to run, but some configuration items of the application systems they run are the same, if you want to modify these identical configuration items, you must modify the PCServer of each running application system at the same time, which is very troublesome and error-prone. Configuration information like this can be managed by Zookeeper, save the configuration information in a directory node of Zookeeper, and then monitor the status of configuration information of all application machines to be modified, once the configuration information changes, each application machine will receive a notification from Zookeeper, and then obtain the new configuration information from Zookeeper and apply it to the system.

3. GroupMembership)

Zookeeper can easily implement cluster management. If multiple servers form a service cluster, you must have a "manager" to know the service status of each machine in the current cluster, once a machine cannot provide services, other clusters in the cluster must know and adjust the re-allocation service policy. When the service capability of the cluster is increased, one or more servers will be added, and the "manager" must also be known .? Zookeeper not only helps you maintain the service status of machines in the current cluster, but also helps you select a "manager" to manage the cluster. This is another Zookeeper function LeaderElection.
? Their implementation method is to create a directory node of the EPHEMERAL type on Zookeeper, and then each Server calls getChildren (String path, boolean watch) on the parent directory node of the directory node they create) and set watch to true. Because it is an EPHEMERAL directory node, when the Server that creates it dies, the directory node is also deleted, so Children will change, watch on getChildren will be called, so other servers will know that a Server is dead. The same principle applies to new servers.
? How to Implement LeaderElection by Zookeeper is to select a MasterServer. As before, each Server creates an EPHEMERAL directory node. The difference is that it is also a SEQUENTIAL directory node, so it is an EPHEMERAL_SEQUENTIAL directory node. It is the EPHEMERAL_SEQUENTIAL directory node because we can number each Server. we can select the Server with the smallest number as the Master node. If the Server with the smallest number dies, because it is an EPHEMERAL node, the node corresponding to the dead Server is also deleted, so there is a node with the smallest number in the current node list, we choose this node as the current Master. In this way, the dynamic choice of the Master is realized, avoiding the problem that a single Master is prone to single point of failure in the traditional sense.

void findLeader() throws InterruptedException {         byte[] leader = null;         try {             leader = zk.getData(root + "/leader", true, null);         } catch (Exception e) {             logger.error(e);         }         if (leader != null) {             following();         } else {             String newLeader = null;             try {                 byte[] localhost = InetAddress.getLocalHost().getAddress();                 newLeader = zk.create(root + "/leader", localhost,                 ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);             } catch (Exception e) {                 logger.error(e);             }             if (newLeader != null) {                 leading();             } else {                 mutex.wait();             }         }     }

4. Locks)

Shared locks are easily implemented in the same process, but they cannot be implemented across processes or between different servers. Zookeeper can easily implement this function. The implementation method is to create an EPHEMERAL_SEQUENTIAL directory node for the Server that needs to obtain the lock, call the getChildren method to obtain whether the smallest directory node in the current directory node list is a self-created directory node. If it is created by itself, it obtains the lock, if not, it calls the exists (String path, boolean watch) method and monitors the changes in the directory node list on Zookeeper until the node created by itself is the directory node with the minimum number in the list, to obtain the lock, it is easy to release the lock, as long as you delete the directory node created by itself.

void getLock() throws KeeperException, InterruptedException{         List list = zk.getChildren(root, false);         String[] nodes = list.toArray(new String[list.size()]);         Arrays.sort(nodes);         if(myZnode.equals(root+"/"+nodes[0])){             doAction();         }         else{             waitForLock(nodes[0]);         }     }     void waitForLock(String lower) throws InterruptedException, KeeperException {        Stat stat = zk.exists(root + "/" + lower,true);         if(stat != null){             mutex.wait();         }         else{             getLock();         }     }

5. Queue Management

Zookeeper can process two types of Queues:
This queue is available only when all the members of a queue are aggregated. Otherwise, it will wait until all the members arrive. This is a synchronization queue.
Queues are queued and queued in FIFO mode, for example, the producer and consumer models are implemented.
The implementation of synchronization queue with Zookeeper is as follows:
Create a parent directory/synchronizing. Each member monitors whether the Set Watch directory/synchronizing/start exists, and each member joins the queue, the queue is created by creating a temporary directory node for/synchronizing/member_ I, and each member obtains all directory nodes in the/synchronizing directory, that is, member_ I. Determine whether the I value is already the number of Members. If it is smaller than the number of members, wait for/synchronizing/start to appear. If it is already equal, create/synchronizing/start.
The following flowchart is easier to understand:

void addQueue() throws KeeperException, InterruptedException{         zk.exists(root + "/start",true);         zk.create(root + "/" + name, new byte[0], Ids.OPEN_ACL_UNSAFE,         CreateMode.EPHEMERAL_SEQUENTIAL);         synchronized (mutex) {             List list = zk.getChildren(root, false);             if (list.size() < size) {                 mutex.wait();             } else {                 zk.create(root + "/start", new byte[0], Ids.OPEN_ACL_UNSAFE,                 CreateMode.PERSISTENT);             }         }  }

When the queue is not full, it enters wait () and waits for the notification from Watch. The Watch code is as follows:

Public void process (WatchedEvent event) {if (event. getPath (). equals (root + "/start") & event. getType () = Event. eventType. nodeCreated) {System. out. println ("Get notification"); super. process (event); doAction ();}}

The implementation of FIFO queues using Zookeeper is as follows:
The implementation idea is also very simple, that is, to create a sub-directory/queue_ I of the SEQUENTIAL type under a specific directory, so that all Members can be added to the queue with numbers, when an outbound queue is sent, the getChildren () method can return the elements in all the current queues, and then consume the smallest one to ensure FIFO.
The following is an example of the queue format for producers and consumers:
Producer code

boolean produce(int i) throws KeeperException, InterruptedException{         ByteBuffer b = ByteBuffer.allocate(4);         byte[] value;         b.putInt(i);         value = b.array();         zk.create(root + "/element", value, ZooDefs.Ids.OPEN_ACL_UNSAFE,                     CreateMode.PERSISTENT_SEQUENTIAL);         return true;     }

Consumer Code

int consume() throws KeeperException, InterruptedException{         int retvalue = -1;         Stat stat = null;         while (true) {             synchronized (mutex) {                 List list = zk.getChildren(root, true);                 if (list.size() == 0) {                     mutex.wait();                 } else {                     Integer min = new Integer(list.get(0).substring(7));                     for(String s : list){                         Integer tempValue = new Integer(s.substring(7));                         if(tempValue < min) min = tempValue;                     }                     byte[] b = zk.getData(root + "/element" + min,false, stat);                     zk.delete(root + "/element" + min, 0);                     ByteBuffer buffer = ByteBuffer.wrap(b);                     retvalue = buffer.getInt();                     return retvalue;                 }             }         }  }

^

Original article address: ZooKeeper Note-Common Operations and applications. Thank you for sharing it with the original author.