Distributed design and development (iii) -- High Consistency service ZooKeeper

In a distributed environment, most services allow partial failures and data inconsistency. However, some of the most basic services require high reliability and consistency, these services are the basis for other distributed services, such as naming service and distributed lock. These distributed basic services have the following requirements:

• High Availability
• High Consistency
• High Performance

Designing a service that challenges the CAP principle is a challenge and a good research topic. Apache's zookeeper may give us a good answer. Zookeeper is a distributed, open-source distributed application Coordination Service that exposes a simple primitive set. distributed applications can implement synchronization services based on it, configuration maintenance and naming services. For more information about zookeeper, see the official documentation.

Basic use of ZooKeeper

It is relatively simple to build a distributed zookeeper environment. The basic steps are as follows:

1) install zookeeper on each server

Download zookeeper and decompress it on each server.

Tar-xzf zookeeper-3.2.2.tar.gz

2) configure the Cluster Environment

Create a configuration file named zoo. cfg under the zookeeper installation directory of each server. Fill in the following content:

 

# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial# synchronization phase can takeinitLimit=10# The number of ticks that can pass between# sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.dataDir=/home/admin/zookeeper-3.2.2/data# the port at which the clients will connectclientPort=2181server.1=zoo1:2888:3888server.2=zoo2:2888:3888

 

Zoo1 and zoo2 correspond to the machine name or IP address of each server in the cluster, and 1 and 2 in server.1 and server.2 correspond to the zookeeper ID of each server, respectively, the method for setting ID is to create a file named myid under the directory configured by datadir and use ID as the file content. In this example, it is set to 1 and 2. For more information about other configurations, see the official documentation.

3) Start the Cluster Environment

Run the zookeeper STARTUP script on each server.

/Home/admin/zookeeper-3.2.2/bin/zkserver. Sh start

4) apply zookeeper

The application zookeeper can execute commands in the shell or call program interfaces in Java or C.

Run the following command in shell:

Bin/zkCli. sh-server 10.20.147.35: 2181

10.20.147.35 indicates the ip address or machine name of any machine in the cluster. After execution, you can go to the zookeeper Operation Panel. For details about how to perform the operation, see the official documentation.

In java, calling program interfaces to apply zookeeper is more complex. You need to understand the concepts of watch and callback. However, this is not required for the simplest CURD experiment, you only need to use the ZooKeeper class. The test code is as follows:

public static void main(String[] args) {try {ZooKeeper zk = new ZooKeeper("10.20.147.35:2181", 30000, null);String name = zk.create("/company", "alibaba".getBytes(),                Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);Stat stat = new Stat();System.out.println(new String(zk.getData(name, null, stat)));zk.setData(name, "taobao".getBytes(), stat.getVersion(), null, null);System.out.println(new String(zk.getData(name, null, stat)));stat = zk.exists(name, null);        zk.delete(name, stat.getVersion(), null, null);        System.out.println(new String(zk.getData(name, null, stat)));} catch (Exception e) {e.printStackTrace();}}

 

The above code is relatively simple. Check the zooKeeper api doc to see how to use it.

Implementation Mechanism of ZooKeeper

The implementation mechanism of ZooKeeper is the most complex open-source framework I have ever seen. It solves the consistency problem in a distributed environment. This scenario also determines the complexity of its implementation. After reading the source code for two or three days, I am still confused. Some of them are beyond my ability. However, by reading the document and other articles written by other experts, I can roughly understand its principles and basic structure.

1) Basic principles of ZooKeeper

ZooKeeper is based on the Fast Paxos algorithm. In the previous blog, we introduced paxos, but we did not mention the existence of live locks in paxos, that is, when multiple proposer are submitted in a staggered manner, they may be mutually exclusive. As a result, no proposer can be submitted successfully, while Fast Paxos has made some optimizations and elected a leader, only the leader can submit propose. The specific algorithm can be seen in Fast.
Paxos. Therefore, to get ZooKeeper, you must first understand Fast Paxos.

2) Basic operation process of ZooKeeper

ZooKeeper has the following two processes:

• Election Leader
• Synchronize data

There are many algorithms in the election Leader process, but the election criteria to be met are consistent:

• The Leader must have the highest zxid
• Most machines in the cluster receive the response and the Leader selected by follow.

The data synchronization process is the essence of zookeeper, and is the specific implementation of the fast paxos algorithm. A cool man draws a zookeeper data flow chart, which intuitively describes how zookeeper synchronizes data.

I cannot understand the essence of the above two core processes at the moment. This is also related to my lack of full understanding of the fast paxos algorithm. Further study is required.

Application field of ZooKeeper

Tim mentioned several main scenarios that can be applied by paxos in his blog, including database replication, naming service, config configuration management, and access control list. This is also the main application scenario of zookeeper. In addition, the zookeeper official document mentions several more basic distributed applications. This is a wonderful use of zookeeper.

1) distributed barrier

Barrier is a mechanism for controlling and coordinating the trigger sequence of multiple tasks. Simply put, it is used to set a gate to block the tasks to be executed. When all tasks are in the executable state, to open the gate. The Mechanism is shown in the following figure:

JDK provides the javasicbarrier class on a single machine to implement this mechanism, but JDK is powerless in a distributed environment. To implement Barrer in a distributed system, high consistency is required. Therefore, Zookeeper can be used. The solution adopted is to use a node as the Barrer entity, A Barrer task needs to call exists () to detect the existence of this node. When barrier needs to be enabled, delete the node. The watch mechanism of zookeeper notifies all tasks to start execution.

2) Distributed Queue

Like Barrier, high consistency is also required to implement Queue in a distributed environment. ZooKeeper provides a simple way to maintain the Queue entity through a Node, its children is used to store Queue content, and the ZooKeeper create method provides an incremental mode. It automatically adds an incremental number after the name to insert new elements. You can use its children to construct a queue data structure. When offer is used, create and take are used to delete the first queue in the order of children. ZooKeeper ensures that the data on each server is consistent, so it implements a distributed Queue. The take and offer instance codes are as follows:

/** * Removes the head of the queue and returns it, blocks until it succeeds. * @return The former head of the queue * @throws NoSuchElementException * @throws KeeperException * @throws InterruptedException */public byte[] take() throws KeeperException, InterruptedException {    TreeMap<Long,String> orderedChildren;    // Same as for element.  Should refactor this.    while(true){        LatchChildWatcher childWatcher = new LatchChildWatcher();        try{            orderedChildren = orderedChildren(childWatcher);        }catch(KeeperException.NoNodeException e){            zookeeper.create(dir, new byte[0], acl, CreateMode.PERSISTENT);            continue;        }        if(orderedChildren.size() == 0){            childWatcher.await();            continue;        }        for(String headNode : orderedChildren.values()){            String path = dir +"/"+headNode;            try{                byte[] data = zookeeper.getData(path, false, null);                zookeeper.delete(path, -1);                return data;            }catch(KeeperException.NoNodeException e){                // Another client deleted the node first.            }        }    }}/** * Inserts data into queue. * @param data * @return true if data was successfully added */public boolean offer(byte[] data) throws KeeperException, InterruptedException{    for(;;){        try{            zookeeper.create(dir+"/"+prefix, data, acl, CreateMode.PERSISTENT_SEQUENTIAL);            return true;        }catch(KeeperException.NoNodeException e){            zookeeper.create(dir, new byte[0], acl, CreateMode.PERSISTENT);        }    }}

 

3) Distributed lock

ZooKeeper is used to implement distributed lock. A Node is used to represent a Lock. When a client obtains the lock, a child with an auto-incrementing sequence is created under the Node, then, use the getChildren () method to check whether the created child is the top child. If yes, the lock is obtained. Otherwise, the exist () method is called to check the second top child, add watch to monitor. When the lock is returned after the lock is executed by the child, only the child created by the lock needs to be deleted. In this case, the watch mechanism notifies all clients that have not obtained the lock, these child will compete for the lock according to the lock rules mentioned above.