Zookeeper cluster configuration and Java Testing Program

Zookeeper cluster configuration and Java Testing Program
Overview

Zookeeper is one of Apache projects and tends to perform collaborative maintenance and management for large applications. IBM gave IBM's knowledge of ZooKeeper: The Zookeeper distributed service framework is a sub-project of Apache Hadoop. It is mainly used to solve some data management problems frequently encountered in distributed applications, such: unified Naming Service, status Synchronization Service, cluster management, and management of distributed application configuration items. In short, the core word "coordination" can be used to describe its role. What can Zookeeper do ?".

Features

We can understand Zookeeper as a streamlined File System (similar to the Linux File System Structure). Each node is called znode, which can store subnodes, you can also directly assign values to the nodes.

Zookeeper is applied to some clusters to improve the high availability of clusters. It helps you avoid spof and make your system more reliable.

The Zookeeper cluster is generally understood as a Leader team with consistent data among all members of the team. The Leader in the team adopts the Election Algorithm recommendation, so it can ensure that when the Leader encounters a problem, it will elect a new Leader. (For details about the fast paxos Election Algorithm)

Zookeeper uses a path to describe a node. A node can be viewed as a directory or a file. It also has the characteristics of both.

The Watch mechanism of Zookeeper is also the biggest reason for its application. When many clients connect to Zookeeper, when the data set for Watch changes, the server sends the change to the client with Watch configured to notify them. Therefore, we often use it for unified configuration management of business systems.Watch with zk should pay special attention to its "one-time trigger" (this was simulated in the last Java example).

Cluster deployment 1. Download

Official Website: http://zookeeper.apache.org/releases.html
Download: zookeeper-3.4.8.tar.gz

2. Installation

Due to limited resources, I created three directories server1, server2, and server3 on the same server to simulate three Server clusters.
Cd server1
Tar-zxvf zookeeper-3.4.8.tar.gz
Mkdir data
Mkdir dataLog
Data is the data directory, and dataLog is the log directory.

3. Configuration

Cd zookeeper-3.4.8/conf
Create the file zoo. cfg with the following content:

tickTime=2000 initLimit=5syncLimit=2dataDir=/opt/zookeeper/server1/datadataLogDir=/opt/zookeeper/server1/dataLogclientPort=2181server.1=127.0.0.1:2888:3888server.2=127.0.0.1:2889:3889server.3=127.0.0.1:2890:3890

TickTime: The Basic time unit used in zookeeper, in milliseconds.
InitLimit: This configuration item is used to configure Zookeeper to accept the client (the client mentioned here is not the client that the user connects to the Zookeeper server, but the Follower server connected to the Leader in the Zookeeper server cluster) the maximum number of tickTime intervals that can be tolerated during connection initialization. The maximum tolerable time for five table names is 5*2000 = 10 seconds.
SyncLimit: This configuration identifies the length of time for sending a message, request, and response between the Leader and Follower. The maximum length is 2*2000 = 4 seconds.
DataDirAndDataLogDirYou can see what the configuration does. You don't need to explain it.
ClientPort: The port number for listening to the client connection. Here, the client is the code program that connects to Zookeeper.
Server. {Myid }={ ip }:{ port on which the leader server exchanges information }:{ when the leader server fails, select the leader port}
MaxClientCnxns: The maximum number of connections on a client is 60 by default, which is sufficient in most cases. However, in actual use, we found that the testing environment often exceeded this number. After investigation, we found that some teams deployed dozens of applications on one machine to facilitate testing, so this number exceeds.

It is very easy to modify zoo. cfg, and you also need to create the myid file:
Cd server1
Echo 1> myid

Copy server1 to server2 and server3, modify the zoo. cfg configuration, and change the content of myid to 2 and 3.

The contents of zoo. cfg of the three servers are as follows:

# server1tickTime=2000initLimit=5syncLimit=2dataDir=/opt/zookeeper/server1/datadataLogDir=/opt/zookeeper/server1/dataLogclientPort=2181server.1=127.0.0.1:2888:3888server.2=127.0.0.1:2889:3889server.3=127.0.0.1:2890:3890

# server2tickTime=2000initLimit=5syncLimit=2dataDir=/opt/zookeeper/server2/datadataLogDir=/opt/zookeeper/server2/dataLogclientPort=2182server.1=127.0.0.1:2888:3888server.2=127.0.0.1:2889:3889server.3=127.0.0.1:2890:3890

# server3tickTime=2000initLimit=5syncLimit=2dataDir=/opt/zookeeper/server3/datadataLogDir=/opt/zookeeper/server3/dataLogclientPort=2183server.1=127.0.0.1:2888:3888server.2=127.0.0.1:2889:3889server.3=127.0.0.1:2890:3890

Here we will explain: Because we simulate a cluster on the same machine, we needNote that the server port and clientPort must not be repeated. Otherwise, a port conflict may occur.. Therefore, if we have three servers on three different machines, then our zoo. cfg configurations are the same (note server. {myid} = the IP address following it uses a specific IP address, for example, 192.168.0.88 ). And,The myid content of each server cannot be the sameThis can also be understood as the identity of different servers.

4. Start

Go to the zookeeper-3.4.8/bin directory and start the zk service with./zkServer. sh start. (You can also use./zkServer. sh start myzoo. cfg to specify the configuration file to start, which is useful in automated O & M)
Use tail-f zookeeper. out to view logs.
It should be said that when the first server is started, there will be a bunch of errors in the log. You can see it carefully because the other two servers have not started the connection failure. Then, when we start the second server, errors in the log will be reduced. Finally, after we start all the servers, there will be no errors in the log.

5. Test

Enter a zk directory and connect to a server for testing.
Cd zookeeper-3.4.8/bin
ZkCli. sh-server 127.0.0.1: 2181
If you want to connect to another server, please specify the specific IP address.

Several Basic commands:
Ls: view the subnodes contained in a specified node (such as ls/or ls/app1/server1)
Create creates a node and assigns a value
Get reads node content
Set changes node content
Delete A node
Note that all nodes in zk are determined based on the path. For example, the command to delete/app1/server1/nodeA is:
Delete/app1/server1/nodeA

The following are basic operations:

Java program Demo

Create a Maven Project
Open the pom. xml file and add the zookeeper dependency.

        
              
   
    org.apache.zookeeper
               zookeeper            
   
    3.4.6
           
  

Create Demo. java with the following code:

Package com. shanhy. demo. zookeeper; import java. io. IOException; import org. apache. zookeeper. createMode; import org. apache. zookeeper. keeperException; import org. apache. zookeeper. watchedEvent; import org. apache. zookeeper. watcher; import org. apache. zookeeper. zooDefs. ids; import org. apache. zookeeper. zooKeeper;/*** Zookeeper Test ** @ create March 10, 2016 */public class Test {// Session Timeout time, which is set to the same as the default private static f time. Inal int SESSION_TIMEOUT = 30*1000; // create a ZooKeeper instance private ZooKeeper zk; // create a Watcher instance private Watcher wh = new Watcher () {/*** Watched event */public void process (WatchedEvent event) {System. out. println ("WatchedEvent >>>" + event. toString () ;}}; // initialize ZooKeeper instance private void createZKInstance () throws IOException {// connect to ZK service, multiple zk = new ZooKeeper ("192.168.19.130: 2181,192.16 8.19.130: 2182,192.168 .19.130: 2183 ", Test. SESSION_TIMEOUT, this. wh);} private void ZKOperations () throws IOException, InterruptedException, KeeperException {System. out. println ("\ n1. create a ZooKeeper node (znode: zoo2, data: myData2, permission: OPEN_ACL_UNSAFE, node type: Persistent"); zk. create ("/zoo2", "myData2 ". getBytes (), Ids. OPEN_ACL_UNSAFE, CreateMode. PERSISTENT); System. out. println ("\ n2. check whether the creation is successful:"); System. Out. println (new String (zk. getData ("/zoo2", this. wh, null); // Add Watch // We added monitoring for the/zoo2 node in the previous line. Therefore, the Watch event is triggered when/zoo2 is modified here. System. out. println ("\ n3. modify node data"); zk. setData ("/zoo2", "shanhy20160310 ". getBytes (),-1); // If you modify it again here, the Watch event will not be triggered. This is one-time triggering feature of ZK, that is, setting one monitoring, it only takes effect for the next operation. System. out. println ('\ n3-1. modify node data again "); zk. setData ("/zoo2", "shanhy20160310-ABCD ". getBytes (),-1); System. out. println ("\ n4. check whether the modification is successful:"); System. out. println (new String (zk. getData ("/zoo2", false, null); System. out. println ("\ n5. delete node"); zk. delete ("/zoo2",-1); System. out. println ("\ n6. check whether the node is deleted:"); System. out. println ("node status: [" + zk. exists ("/zoo2", false) + "]");} private void ZKClose () throws InterruptedException {zk. close ();} public static void main (String [] args) throws IOException, InterruptedException, KeeperException {Test dm = new Test (); dm. createZKInstance (); dm. ZKOperations (); dm. ZKClose ();}}

I don't think the code needs to be explained. The comments in the comments are all commented out.