Examples of zookeeper introductory lectures

Probe into the use and principle of zookeeper (I.)

ZookeeperIntroduction
Zookeeper is a software that provides consistent services for distributed applications, a subproject in an open source Hadoop project, and based on the <the Chubby lock service for loosely-coupled published by Google Distributed systems> paper to achieve, next we first to install the use of the software, and then to explore the more important consistency algorithm.

Zookeeper Installation and use
Zookeeper installation can basically follow the steps on http://hadoop.apache.org/zookeeper/docs/current/zookeeperStarted.html this page to complete the installation, Here are the main steps to deploy a cluster, because this official page doesn't seem to be very detailed (Running replicated Zookeeper).

Because there are not enough machines on hand, you can do this by deploying 3 servers on a single machine, if you have a tighter hand. Then I have built 3 folders, as follows
Server1 Server2 Server3

Then in each folder to extract a zookeeper download package, and also built a few folders, the overall structure is as follows, the last is to download the compressed package to extract files
Data DataLog Logs zookeeper-3.3.2

Then first go to the data directory, create a myID file, write a number, such as my this is server1, then write a 1,server2 corresponding myID file write 2,server3 corresponding myID file write a 3

then enterzookeeper-3.3.2/confDirectory, then if it is just coming down, there will be 3 files,Configuration.xml, Log4j.properties,zoo_sample.cfg, the first thing we need to do in these 3 files is to create a zoo.cfg configuration file in this directory, and of course you can change the zoo_sample.cfg file to Zoo.cfg, and the contents of the configuration are as follows:
ticktime=2000
Initlimit=5
synclimit=2
Datadir=xxxx/zookeeper/server1/data
Datalogdir=xxx/zookeeper/server1/datalog
clientport=2181
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

Some of the red configuration should be made clear on the official website, just need to note that clientport this port if you are deploying multiple servers on 1 machines, then each machine needs a different clientport, such as my server1 is 2181, Server2 is 2182,server3 is 2183,datadir and Datalogdir also need to differentiate.

The only thing to notice in the last few lines is the server. X this number is the corresponding number in the Data/myid. You write the myID file in the 3 server, respectively, the zoo.cfg in each server will be OK with server.1,server.2,server.3. Because on the same machine, the next 2 ports attached to 3 servers are not the same, or port conflicts, where the first port is used for the information exchange of the cluster members, the second port is used specifically for election leader when leader is hung out.

Enter the Zookeeper-3.3.2/bin directory,./zkserver.sh StartStart a server and then report a large number of errors? It doesn't really matter, because the cluster now has only 1 servers, Zookeeper server will be based on the ZOO.CFG server list to initiate the election leader request, because not connected to other machines and error, then when we start the second zookeeper instance, leader will be selected, so that the consistency service can start to use, because 3 machines as long as 2 units available You can select leader and provide services (2n+1 machine, can be used to hold n machines hanging off).

Then you can use it, and we can start by zookeeper the client interaction program to simply feel what the zookeeper is doing. Enter Zookeeper-3.3.2/bin (any one of 3 servers),./zkcli.sh–server 127.0.0.1:2182, I even have a machine with 2182 ports open.

Well, first we make a random order, because zookeeper do not know, he will give the command of help, such as

LS (view current node data),
LS2 (view current node data and see the number of updates, etc.),
Create (creates a node),
Get (get a node that contains data and the number of updates),
Set (Modify node)
Delete (remove a node)

Through the above command practice, we can find that zookeeper uses a similar file system tree structure, the data can be hung on a node, you can make a deletion of this node. We also found that when a node is changed, the surviving machines in the cluster will be updated to the same data.

ZookeeperThe data Model
After simply using zookeeper, we found that the data model is somewhat like the operating system's file structure, as shown in



(1) Each node is called the Znode in zookeeper, and it has a unique path identifier, such as the/server2 node identity is/app3/server2
(2) Znode can have sub-znode, and Znode can save data, but nodes of ephemeral type cannot have child nodes
(3) The data in the Znode can have more than one version, such as a path to have multiple versions of the data, then query the path of the data will need to bring the version.
(4) Znode can be a temporary node, once created this Znode client and the server lost contact, the Znode will be automatically deleted, Zookeeper client and server communication using long connection, each client and server through the heartbeat to maintain connectivity, this connection status is called Session, if Znode is a temporary node, this session expires, Znode also deleted
(5) Znode directory name can be automatically numbered, such as APP1 already exist, and then created, will be automatically named as App2
(6) znode can be monitored, including changes in the data stored in this directory node, changes in sub-node directories, and so on, once the change can notify the settings monitoring client, this function is zookeeper for the application of the most important features, through this feature can be implemented by the functions including the centralized management of the configuration, Cluster management, distributed locks and so on.

using Zookeeper with Java code
The use of zookeeper is mainly through the creation of the zookeeper instance under its jar package, and calls its interface method, the main operation is to znode the deletion and modification operation, monitoring the change of znode and processing.

The following are the main API usage and interpretation

Create a ZooKeeper instance, the first parameter is the destination server address and port, the second parameter is the session time-out time, the third is the callback method when the node changes ZooKeeper ZK = new ZooKeeper ("127.0.0.1:2181", 500000,new Watcher () {           //monitor all triggered events public             void process (Watchedevent event) {           //dosomething           }      }) ;//Create a node root, the data is MyData, no ACL permission control, the node is permanent (that is, the client shutdown will not disappear) zk.create ("/root", "MyData". GetBytes (), Ids.open_ Acl_unsafe, createmode.persistent);//Create a childone znode under root, the data is Childone, do not control ACL permissions, the node is a permanent zk.create ("/root/ Childone "," Childone ". GetBytes (), ids.open_acl_unsafe,createmode.persistent);//Get the name of the child node under the/root node, return to List<string >zk.getchildren ("/root", true);//Gets the data under the/root/childone node, returns Byte[]zk.getdata ("/root/childone", true, null);// Modify the data under Node/root/childone, the third parameter is version, if 1, that will ignore the modified version of the data, directly change the Zk.setdata ("/root/childone", "Childonemodify". GetBytes () ,-1);//delete/root/childone This node, the second parameter is version, 1 words are deleted directly, ignoring version zk.delete ("/root/childone",-1);      Close Sessionzk.close ();

Zookeeper of the mainstream application scenario (except for the official example)

(1)Configuration Management
Centralized configuration management is common in application clusters, where a centralized set of configuration management centers is implemented within a common business company, responding to the need for different application clusters to share their respective configurations and being able to notify every machine in the cluster when configuration changes are made.

Zookeeper is easy to implement this centralized configuration management, such as the configuration of all APP1 configuration to/app1 Znode, APP1 all the machines start on the/APP1 this node monitoring (Zk.exist ("/app1", true)), and implement the callback method watcher, then on zookeeper/app1 znode node under the data changes, each machine will be notified, the Watcher method will be executed, then the application can then remove the data (Zk.getdata ("/app1", False,null));

The above example is simply coarse granular configuration monitoring, fine-grained data can be monitored hierarchically, all of which can be designed and controlled.
(2) Cluster management
In an application cluster, we often need to let each machine know which machines are alive in the cluster (or some other cluster depending on them), and can quickly notify every machine in the event that the cluster machine is not manually involved due to outages, network disconnection, etc.

Zookeeper is also very easy to implement this function, such as I have a zookeeper server side znode called/app1servers, then every machine in the cluster when the start of the node to create a ephemeral type of node, For example Server1 create/APP1SERVERS/SERVER1 (can use IP, guaranteed not to repeat), Server2 create/app1servers/server2, then SERVER1 and SERVER2 Watch/ App1servers This parent node, then the data or child node changes under the parent node will notify the client of the watch on that node. Because the ephemeral type node has a very important feature, that is, the client and server side connection is broken or the session expires will cause the node to disappear, then when a machine hangs or breaks the chain, the corresponding node will disappear, and then the cluster of all the The client that App1servers watch will be notified and then get the latest list.

Another application scenario is the cluster select Master, once master hangs off can immediately select a master from the slave, the implementation of the same steps as the former, but the machine at the start of the app1servers created in the node type changed to Ephemeral_ Sequential type, so that each node is automatically numbered, for example

 Zk.create ("/testrootpath/testchildpath1", "1". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral_sequential);        Zk.create ("/testrootpath/testchildpath2", "2". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral_sequential);        Zk.create ("/testrootpath/testchildpath3", "3". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral_sequential); Create a subdirectory node zk.create ("/testrootpath/testchildpath4", "4". GetBytes (), Ids.open_acl_unsafe,createmode.ephemeral_ sequential); System.out.println (Zk.getchildren ("/testrootpath", false)); Printed results: [testChildPath10000000000, testChildPath20000000001, testChildPath40000000003, testChildPath30000000002] Zk.create ("/testrootpath", "Testrootdata". GetBytes (), Ids.open_acl_unsafe, createmode.persistent);//        Create a subdirectory node zk.create ("/testrootpath/testchildpath1", "1". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral);        Zk.create ("/testrootpath/testchildpath2", "2". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral); Zk.create ("/testrootpath/testchIldPath3 "," 3 ". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral); Create a subdirectory node zk.create ("/testrootpath/testchildpath4", "4". GetBytes (), ids.open_acl_unsafe,createmode.ephemeral); System.out.println (Zk.getchildren ("/testrootpath", false));

Printed results: [TestChildPath2, TestChildPath1, TestChildPath4, TestChildPath3]

We default to the minimum number of the master, so when we monitor the/app1servers node, we get a list of servers, as long as all the cluster machine logic that the minimum number node is master, then Master is selected, and this master down, The corresponding znode disappears, and then the new server list is pushed to the client, then each node logic considers the minimum number node as master, so that the dynamic master election is done.


Summary

We initially used the zookeeper and tried to describe the specific implementation of several scenarios, and then we will try to explore zookeeper's high availability and leaderelection algorithms.

Reference : http://www.ibm.com/developerworks/cn/opensource/os-cn-zookeeper/

http://hadoop.apache.org/zookeeper/docs/current/

Examples of zookeeper introductory lectures