Apache Open Source Project--zookeeper

Zookeeper is a full-fledged sub-project of Hadoop, a reliable coordination system for large distributed systems, with features such as configuration maintenance, name services, distributed synchronization, group services, and more. The goal of zookeeper is to encapsulate complex and error-prone services that provide users with easy-to-use interfaces and performance-efficient, robust systems.

Zookeeper is an open source implementation of Google's chubby. is a highly effective and reliable collaborative work system. Zookeeper can be used to leader elections, configure information maintenance, etc. in a distributed environment, we need a master instance or store some configuration information , ensuring consistency in file writes, and so on. Zookeeper can guarantee the following 3 points:

• Watches is ordered with respect to other events, other Watches, and
  Asynchronous replies. The ZooKeeper client libraries ensures that
  Everything is dispatched in order.

• A client would see a watch event for a znode it was watching before seeing the new data that corresponds to that znode.

• The Order of watch events from ZooKeeper corresponds to the order of the updates as seen by the ZooKeeper service.

In zookeeper, Znode is a node similar to a UNIX file system path that can store or fetch data to that node. If flag is set to ephemeral when the Znode is created, Then when the node that created this znode and zookeeper loses connection, the Znode will no longer exist in zookeeper. Zookeeper uses watcher to detect event information when the client receives event information, such as a connection timeout, node data changes, Child node changes, you can call the corresponding behavior to process the data. Zookeeper's wiki page shows how to use zookeeper to handle event notifications, queues, priority queues, locks, shared locks, revocable shared locks, and two-phase commits.

So what can zookeeper do for us? Simple example: Let's say we have a 20-search engine server (each of the search tasks responsible for a subset of the total index) and a total server (responsible for making search requests to the servers of these 20 search engines and merging result sets). An alternate total server (responsible for replacing the total server when the total server goes down), a web CGI (a search request to the general server). 15 servers in the search engine now provide a search service and 5 servers are building indexes. The servers of these 20 search engines often make The server that provides the search service stops providing services to start building the index, or the server that generated the index has completed indexing to provide services. Using zookeeper can ensure that the total server automatically senses how many servers are providing search engines and makes searches to those servers. When an alternate total server is down, the standby total server is automatically enabled, and the Web CGI automatically learns the network address changes of the total server. How do we do that?

1. The server that provides the search engine creates znode,zk.create ("/search/nodes/node1" in Zookeeper,
   "Hostname". GetBytes (), Ids.open_acl_unsafe, createflags.ephemeral);

2. The total server can get a list of Znode child nodes from Zookeeper, Zk.getchildren ("/search/nodes", true);

3. The total server traverses these child nodes and gets a list of the servers that provide the search engine for the child node's data generation.

4. When the total server receives the event information changed by the child node, return to the second step again.

5. The total server creates nodes in Zookeeper, Zk.create ("/search/master", "hostname". GetBytes (), Ids.open_acl_unsafe, createflags.ephemeral );

6. The standby total server monitors the "/search/master" node in the zookeeper. When the Znode node data changes, it turns itself into a total server and puts its network address data into this node.

7. The CGI of the web obtains the network address data of the total server and sends a search request to it from the "/search/master" node in zookeeper.

8. The "/search/master" node in the web's CGI monitoring zookeeper, when this znode node data changes, obtains the network address data of the total server from this node and changes the current total server's network address.

Apache Open Source Project--zookeeper