Zookeeper (vi) application examples

6.1 Java API

The client to connect to the zookeeper server can interact with the server by creating an instance object of Org.apache.zookeeper.ZooKeeper and then invoking the interface provided by this class.

ZooKeeper is primarily used to maintain and monitor the state of the data stored in a directory node tree, all of which we can manipulate ZooKeeper and manipulate the directory node tree in much the same way, such as creating a directory node, setting up data for a directory node, getting all the subdirectory nodes of a directory node, Set permissions for a directory node and monitor the status changes for this directory node.

Here is a code example to familiarize yourself with the common methods of JAVAAPI.

Import Java.util.list;import Org.apache.zookeeper.createmode;import Org.apache.zookeeper.watchedevent;import Org.apache.zookeeper.watcher;import Org.apache.zookeeper.zoodefs.ids;import Org.apache.zookeeper.ZooKeeper; Import Org.apache.zookeeper.data.stat;public class Zktest {private static final String connect_string = "127.0.0.1:2181" ;p rivate static final int session_timeout = 3000;public static void Main (string[] args) throws Exception {//define a monitor for all node changes Watcherwatcher allchangewatcher = new Watcher () {@Overridepublic void process (Watchedevent event) {System.out.println (" **watcher receive watchedevent** changed path: "+ event.getpath () +"; Changed type: "+ event.gettype (). name ());}};/ /Initialize one with ZK connection. Three parameters://1, the server address to connect, "ip:port" format;//2, Session timeout//3, Node change monitor ZooKeeper ZK = new ZooKeeper (connect_string, Session_timeout, Allchangewatcher);//New node. Four parameters: 1, node path, 2, node data, 3, node permissions, 4, create mode zk.create ("/myname", "Chenlongfei". GetBytes (), Ids.open_acl_unsafe, Createmode.persistent); System.out.println ("CreateNew node '/myname ');//Determine if a path exists. Two parameters: 1, node path, 2, whether to monitor (watcher that is, the incoming watcher when initializing zookeeper) Stat Beforsstat = zk.exists ("/myname", true); System.out.println ("Stat of '/myname ' before change:" + beforsstat.tostring ());//Modify node data. Three parameters: 1, node path, 2, new data, 3, version, if 1, match any version of stat Afterstat = Zk.setdata ("/myname", "CLF". GetBytes (),-1); System.out.println ("Stat of '/myname ' after change:" + afterstat.tostring ());//Gets all child nodes. Two parameters: 1, node path, 2, whether to monitor the node list<string> children = Zk.getchildren ("/", true); System.out.println ("Children of Path '/':" + children.tostring ());//Get node data. Three parameters: 1, the node path, 2, the book does not monitor the node, 3, version and other information can be specified by a stat object byte[] namebyte = Zk.getdata ("/myname", true, NULL); String name = new String (Namebyte, "UTF-8"); SYSTEM.OUT.PRINTLN ("Get Data from '/myname ':" + name);//delete node. Two parameters: 1, node path, 2, version, 1 can match any version, will delete all data Zk.delete ("/myname", 1); System.out.println ("Delete '/myname '"); Zk.close ();}

Run the program and print the results as follows:

For more detailed API please refer to the official website.

Zookeeper from a design pattern perspective, it is a distributed service management framework based on the observer pattern design that stores and manages the data that everyone cares about and then accepts the viewer's registration, and once the status of these data changes, Zookeeper will be responsible for notifying the Zookeeper The observers who are registered on the register respond accordingly.

The following is a typical use of two zookeeper scenarios to understand the characteristics of the zookeeper and use of the method.

6.2 Distributed Locks

Let's review the lock control in multi-threading.

public class Multithreadtest {//with a static variable to simulate common resources private static int counter = 0;//multi-threaded environment, concurrency problem is present in the general static void Plus () { The counter plus a counter++;//thread randomly sleeps for milliseconds, simulating the time-consuming operation of the reality int sleepmillis = (int) (Math.random () *); try {thread.sleep (sleepmillis);} catch (Interruptedexception e) {e.printstacktrace ();}} Thread implementation class Countplus extends thread {@Overridepublic void Run () {for (int i = 0; i <; i++) {Plus ();} System.out.println (Thread.CurrentThread (). GetName () + "execution Completed:" + counter);} Public Countplus (String threadname) {super (threadname);}} public static void Main (string[] args) throws Exception {//Open five threads countplus Threada = new Countplus ("Threada"); Threada.sta RT (); Countplus threadb = new Countplus ("threadb"); Threadb.start (); Countplus THREADC = new Countplus ("THREADC"); Threadc.start (); Countplus Threadd = new Countplus ("Threadd"); Threadd.start (); Countplus Threade = new Countplus ("Threade"); Threade.start ();}}

In the above example, five threads are turned on, and each thread adds 20 times to the static variable counter by the plus () method, and the expected counter will eventually become 100. To run the program:

It can be found that after five threads have been executed, the counter does not become a 100. The plus () method involves changes to the public resources, but does not control it synchronously, which can cause multiple threads to initiate changes to the public resources at the same time, resulting in concurrency problems. The root of the problem is that in the previous example there is no guarantee that only one thread can change public resources at the same time.

To add the synchronized keyword to the plus () method, rerun the program:

Visible, and the expected results are achieved.

The role of the Synchronized keyword is to add lock control to the plus () method, a thread that wants to execute the method, first to acquire a lock (the lock is unique), and then release the lock after execution is complete. If the lock is not reached, the thread waits in the waiting pool until the lock is taken to continue. This ensures that only one thread can change the public resources at the same time, avoiding concurrency problems.

Shared locks are easy to implement in the same process, and can be solved by the synchronization mechanism provided by Java itself, but not in a cross-process or between different servers, requiring an intermediary to coordinate various issues between multiple servers, such as how to get a lock/release lock, who gets the lock first, Who gets locks and so on later.

This distributed lock can be implemented with zookeeper: the Server that needs to obtain the lock creates a ephemeral_sequential directory node, and then calls the GetChildren () method to get the smallest directory node in the list. If the minimum node is the directory node that you created, then it gets the lock, and if it is not then it calls the exists () method and monitors the changes of the previous node, until the node that it creates becomes the smallest numbered directory node in the list, thereby acquiring the lock. Releasing the lock is simple, just delete the directory node that you created.

The flowchart is as follows:

Below we transform the code just now, do not synchronize the keyword but use zookeeper to achieve the purpose of lock control, simulate the realization of the distributed lock.

Import Java.util.collections;import Java.util.list;import Org.apache.zookeeper.createmode;import Org.apache.zookeeper.keeperexception;import Org.apache.zookeeper.watchedevent;import Org.apache.zookeeper.watcher;import Org.apache.zookeeper.zoodefs.ids;import Org.apache.zookeeper.ZooKeeper; Import Org.apache.zookeeper.data.stat;public class Zkdistributedlock {//with a static variable to simulate public resources private static int counter = 0; public static void Plus () {///counter plus one counter++;//thread random sleep milliseconds, simulates the time-consuming operation of the reality int sleepmillis = (int) (Math.random () *); try {Thr Ead.sleep (Sleepmillis);} catch (Interruptedexception e) {e.printstacktrace ();}} Thread implementation class Countplus extends thread {private static final String Lock_root_path = "/locks";p rivate static final  String lock_node_name = "Lock_";//each thread holds a ZK client, is responsible for acquiring lock and release lock zookeeper zkclient; @Overridepublic void Run () {for (int i = 0; i < 20; i++) {//To access the counter before you need to obtain the lock string path = Getlock ();//Perform task plus ();//Release the lock ReleaseLock (path) after executing the task;} Closezkclient (); System.out.println (Thread.curreNtthread (). GetName () + "execution Completed:" + counter);} /** * Acquires a lock, that is, creates a child node, acquires the lock when the node becomes the node with the smallest ordinal number */private string Getlock () {try {//Create Ephemeral_sequential type node string lockpath = Zkcli Ent.create (Lock_root_path + "/" + Lock_node_name,thread.currentthread (). GetName (). GetBytes (), Ids.open_acl_unsafe, Createmode.ephemeral_sequential); System.out.println (Thread.CurrentThread (). GetName () + "Create path:" + Lockpath);//attempt to acquire lock Trylock (Lockpath); return Lockpath;} catch (Exception e) {e.printstacktrace ();} return null;} /** * This function is a recursive function if the lock is obtained, it is returned directly; otherwise, the blocking thread waits for the previous node to release the lock message and then re-trylock the */private boolean trylock (String Lockpath) throws Keeperexception, Interruptedexception {//Get all child nodes under Lock_root_path and Sort by node ordinal list<string> lockpaths = Zkclient.getchildren (Lock_root_path, false); Collections.sort (lockpaths); int index = Lockpaths.indexof (lockpath.substring (Lock_root_path.length () + 1)); if (index = = 0) {//Lockpath is the node with the smallest ordinal, gets the lock System.out.println (Thread.CurrentThread (). GetName () + "Get lock, Lockpath:" + Lockpath); Return true;} else {//Lockpath is not a node with the smallest ordinal number//create Watcher, monitoring the previous node of Lockpath Watcher Watcher = New Watcher () {@Overridepublic void process (Wat Chedevent event) {System.out.println (Event.getpath () + "has been deleted"), synchronized (this) {Notifyall ();}}; String Prelockpath = Lockpaths.get (index-1); Stat stat = zkclient.exists (Lock_root_path + "/" + Prelockpath, Watcher), if (stat = = null) {///For some reason, the previous node does not exist (for example, if the connection is broken), heavy New Trylockreturn Trylock (Lockpath);} else {//blocks the current process until Prelockpath releases the lock, re-tryLockSystem.out.println (Thread.CurrentThread (). GetName () + "Wait for" + Prelockpath); synchronized (watcher) {watcher.wait ();} Return Trylock (Lockpath);}} /** * Release the lock, that is, delete the Lockpath node */private void ReleaseLock (String lockpath) {try {zkclient.delete (Lockpath,-1);} catch (Interrup tedexception | Keeperexception e) {e.printstacktrace ();}} public void Setzkclient (ZooKeeper zkclient) {this.zkclient = zkclient;} public void Closezkclient () {try {zkclient.close ();} catch (Interruptedexception e) {e.printstacktrace ()}} Public Countplus (String threadname) {super (threadname);}} public static void Main (string[] args) throws Exception {//Open five threads countplus Threada = new Countplus ("Threada"); setzkclient (Threada); Threada.start (); Countplus threadb = new Countplus ("threadb"); Setzkclient (THREADB); Threadb.start (); Countplus THREADC = new Countplus ("THREADC"); Setzkclient (THREADC); Threadc.start (); Countplus Threadd = new Countplus ("Threadd"); Setzkclient (Threadd); Threadd.start (); Countplus Threade = new Countplus ("Threade"); Setzkclient (Threade); Threade.start ();} public static void Setzkclient (Countplus thread) throws Exception {ZooKeeper zkclient = new ZooKeeper ("127.0.0.1:2181", 30 , null); Thread.setzkclient (zkclient);}}

Note : You need to create "/locks" as the root node for holding the lock information before running the program.

Once a server wants to acquire a lock, it creates a ephemeral_sequential type named "Lock_" under/locks, and zookeeper automatically appends an incremented to each child node . Number, which is of type int, length 10, left with 0 complement. A series of nodes will be maintained under "/locks":

lock_0000000001,lock_0000000002, lock_0000000003, lock_0000000004 ...

Once these nodes are created, the server is disconnected and the node is purged (and, of course, actively purged).

Because the number of nodes is incremented, the later the ranking is created, the higher the post. On a first-come-first-served basis, after the server has created the node, it will check if its node is the smallest, and if so, get the lock, if not, wait in line. After performing the task, the server clears the node that it created so that subsequent nodes get the lock in turn.

The running results of the program are as follows:

6.3 Distributed queues

A lot of ordinary things on a single machine, placed in the cluster environment will be a qualitative change.

Take a common producer-consumer model example: there is a mailbox with a limited capacity, and the sender (i.e. the producer) keeps the letter in the mailbox, and the postman (i.e. the consumer) constantly pulls the letter out of the mailbox and sends it to the destination. During operation, you need to ensure that:

(1) When the mailbox has reached the upper limit, the sender stops the activity, and so on with the mailbox restored to the non-full status

(2) When the mailbox is empty, the postman stops the activity and waits for the mailbox to return to a non-empty state.

The mailbox is implemented in an orderly queue to ensure FIFO (first-in-a-out) features.

On a single machine, you can use an orderly queue to implement the mailbox, guaranteed FIFO (first-in and out) features, open two threads, one to act as a sender, a postman, through Wait ()/notify () easy to achieve the above functions.

But what if in a cross-process or distributed environment? For example, one machine runs a producer program, another machine runs a consumer program, and an ordered queue representing a mailbox cannot be shared across machines, but both need to be aware of the status of the mailbox (whether it is full, empty), and to ensure that the letter is ordered (first arrived first sent).

In this case, a distributed queue can be implemented with zookeeper. Create a new "/mailbox" node to represent the mailbox. Once a letter arrives, a child node of type persistent_sequential is created under that node, blocking the producer when the total number of child nodes reaches the upper limit, and then using GetChildren (String path, watcher Watcher) method to monitor the changes of child nodes, the total number of sub-nodes to reduce and then reply to production, and the consumer selects the smallest number of child nodes to process, and then delete the node, when the total number of child nodes is 0 o'clock, blocking consumers, the same set of monitoring, the total number of child nodes and then reply to consumption.

The code is as follows:

Import Java.io.ioexception;import Java.util.collections;import Java.util.list;import Org.apache.zookeeper.createmode;import Org.apache.zookeeper.keeperexception;import Org.apache.zookeeper.watchedevent;import Org.apache.zookeeper.watcher;import Org.apache.zookeeper.ZooDefs.Ids; Import Org.apache.zookeeper.zookeeper;import Org.apache.zookeeper.data.stat;public class Zkdistributedqueue {// Mailbox limit is 10 letter private static final int mailbox_max_size = 10;//mailbox Path private static final String Mailbox_root_path = "/mailbox"; The letter node private static final String Letter_node_name = "letter_";//producer Thread, responsible for accepting the letter static class Producer extends thread {Zooke Eper zkclient; @Overridepublic void Run () {while (true) {try {if (getletternum () = = Mailbox_max_size) {//mailbox is full SYSTEM.OUT.P Rintln ("MailBox have been full");//Create Watcher, monitor child node changes Watcher Watcher = new Watcher () {@Overridepublic void process (Watche Devent event) {//producer has stopped, only the consumer is active, so only the action of sending the letter System.out.println ("MailBox have been not full"); synchronized (this) { Notify(); Wake producer}}};zkclient.getchildren (Mailbox_root_path, watcher); synchronized (watcher) {watcher.wait ();//Block producer}} else { The thread randomly sleeps for several milliseconds, simulating the time-consuming operation of the reality int sleepmillis = (int) (Math.random () * 1000); Thread.Sleep (Sleepmillis);//Receive a letter, create a new child node string newletterpath = Zkclient.create (Mailbox_root_path + "/" + Letter_node_ NAME, "letter". GetBytes (), Ids.open_acl_unsafe, createmode.persistent_sequential);  System.out.println ("A new letter have been received:" + newletterpath.substring (mailbox_root_path.length () +1) + ", Letter Num: "+ getletternum ());}} catch (Exception e) {System.out.println ("producer Equit task becouse of Exception!"); E.printstacktrace (); break;}}} private int Getletternum () throws Keeperexception, interruptedexception {stat stat = zkclient.exists (Mailbox_root_path, NULL); int letternum = Stat.getnumchildren (); return letternum;} public void Setzkclient (ZooKeeper zkclient) {this.zkclient = zkclient;}} Consumer thread, responsible for sending the letter static class Consumer extends thread {ZooKeeper zkclient; @Overridepublic voID run () {while (true) {try {if (getletternum () = = 0) {//mailbox is empty SYSTEM.OUT.PRINTLN ("MailBox has been empty");//Create Watcher, monitor Change of the control node Watcher watcher = New Watcher () {@Overridepublic void process (Watchedevent event) {//consumer stopped, only producer in activity, So it is possible to receive the action of the letter System.out.println ("MailBox have been not empty"), synchronized (this) {notify ();//wake-up Consumer}}; Zkclient.getchildren (Mailbox_root_path, watcher); synchronized (watcher) {watcher.wait ();//Blocking consumer}} else {//thread random sleep milliseconds , simulating time-consuming operations in reality int sleepmillis = (int) (Math.random () * 1000); Thread.Sleep (Sleepmillis);//Send a letter, delete the smallest sub-node of the ordinal string firstletter = Getfirstletter (); Zkclient.delete (mailbox_root_ PATH + "/" +firstletter,-1); SYSTEM.OUT.PRINTLN ("A letter had been delivered:" + firstletter + ", Letter num:" + getletternum ());}} catch (Exception e) {System.out.println ("Consumer equit task becouse of Exception!"); E.printstacktrace (); break;}}} private int Getletternum () throws Keeperexception, interruptedexception {stat stat = zkclient.exists (Mailbox_root_path, FALSE);int letternum = Stat.getnumchildren (); return letternum;} Private String Getfirstletter () throws Keeperexception, interruptedexception {list<string> letterpaths = Zkclient.getchildren (Mailbox_root_path, false); Collections.sort (letterpaths); return letterpaths.get (0);} public void Setzkclient (ZooKeeper zkclient) {this.zkclient = zkclient;}} public static void Main (string[] args) throws IOException {//Turn on producer thread producer producer = new producer (); ZooKeeper Zkclienta = new ZooKeeper ("127.0.0.1:2181", n/A, NULL);p roducer.setzkclient (Zkclienta);p Roducer.start (); /Open consumer thread consumer consumer = new Consumer (); ZooKeeper ZKCLIENTB = new ZooKeeper ("127.0.0.1:2181", n/A, NULL); Consumer.setzkclient (ZKCLIENTB); Consumer.start ();}}

The printing results are as follows:

There is also an improvement in the above example, in a distributed environment, such constants like Mailbox_max_size are shared by multiple machines, and may change during operation, such as the mailbox limit from 10 to 20, only to stop the machine, and then change the parameters on each machine, and then redeploy. However, what if the service does not allow downtime and is deployed on dozens of machines, allowing parameters to take effect at runtime and consistent?

This involves another typical application scenario for zookeeper-the configuration center. The parameters shared by multiple machines can be hosted on the Znode, the machine that is concerned about this parameter is registered watcher on Znode, once the parameter changes, the registrant will receive the message, and then make the corresponding adjustment.

The role of zookeeper, of course, more than this, more applications need users in the actual project excavation and exploration, after all, the end of the paper, the practice of knowledge.

Zookeeper (vi) application examples