The application of the six-zookeeper of zookeeper series

Http://www.cnblogs.com/sharpxiajun/archive/2013/06/02/3113923.html

Zookeeper is a sub-project of Hadoop, and although it originates from Hadoop, I find that there are more and more applications to develop a distributed framework zookeeper out of the scope of Hadoop. Today I want to talk about zookeeper, this article does not talk about how to use zookeeper, but zookeeper exactly what practical applications, those types of applications can play the advantages of zookeeper, finally talk about the zookeeper on the distributed Web site architecture can produce what role.

Zookeeper is a highly reliable coordination system for large-scale distributed systems. By this definition, we know that zookeeper is a coordination system, and the object is distributed system. Why does a distributed system need a coordination system? The reasons are as follows:

Developing a distributed system is a difficult task, and the difficulty lies in the "partial failure" of the distributed system. "Partial failure" refers to the transmission of information between the two nodes of the network, if the network fails, the sender can not know whether the recipient received this information, and the cause of this failure is very complex, the recipient may have received a network error before the message, or may not receive, or the receiver's process died. The only way for the sender to get to the real situation is to reconnect to the recipient and ask the recipient for the reason for the error, which is the "partial failure" issue in Distributed system development.

Zookeeper is the framework for solving the "partial failure" of distributed systems. Zookeeper not let the distributed system avoid "partial failure" problem, but let the distributed system when encountering partial failure, can correctly handle such problems, so that the distributed system can run normally.

Next I would like to talk about the zookeeper driver application scenario:

Scenario One: There is a group of servers to provide a service to the client (for example: the service side of the distributed Web site I have previously done, which is a cluster of four servers, to provide services to the front-end cluster), we want the client to find a server in the service-side cluster every time the service is requested. This allows the server to provide the client with the services required by the client. For this scenario, there must be a list of these servers in our program, and each time the client requests it, it reads the list of servers from this table. Then this list obviously cannot be stored on a single node of the server, otherwise this node hangs up, the whole cluster will fail, we hope this list is highly available. The highly available solution is that the list is distributed storage, which is managed by the server that stores the list, and if one of the servers in the table is broken, the other server can immediately replace the broken server, and the broken server can be removed from the table, leaving the failed server out of the cluster , and all of this does not operate by the failed server, but the normal server in the cluster. This is an active distributed data structure that proactively modifies the state of data items when external conditions change. This service is provided by the zookeeper framework. The service name is the unified naming service, which resembles the Jndi service in Java EE.

Scenario Two: Distributed lock service. When the distributed system operates data, for example: reading data, analyzing data, and finally modifying data. In the distributed system, these operations may be dispersed to different nodes in the cluster, then there is the problem of consistency in the data operation process, if not consistent, we will get a wrong result, in a single process program, the consistency of the problem is very good solution, but to the distributed system is more difficult, Because the operation of the different servers in the distributed system is in the independent process, the intermediate result and the process of the operation also pass through the network, so it is more difficult to achieve the consistency of data operation. Zookeeper provides a lock service that solves this problem, allowing us to ensure consistency of data operations while doing distributed data operations.

Scenario Three: Configuration management. In the distributed system, we will deploy a service application to n servers, the configuration files are the same (for example: I designed the distributed site framework, the server has 4 servers, 4 servers are the same, the configuration files are the same), If the configuration options change, then we have to go to those configuration files, if we need to change the number of servers less, these operations are not too cumbersome, if we have more distributed servers, such as some large internet companies Hadoop cluster has thousands of servers, Changing configuration options is a cumbersome and dangerous thing to do. This time zookeeper can come in handy, we can use zookeeper as a high-availability configuration memory, to the zookeeper to manage such things, we copy the cluster configuration file to a node of the zookeeper file system, And then with the zookeeper the state of the configuration files in all distributed systems, once the configuration file has been found to change, each server will be zookeeper comrades, let each server sync zookeeper configuration files, The Zookeeper service also guarantees the atomicity of the synchronization operation, ensuring that each server's configuration file is updated correctly.

Scenario Four: Provides fault-repair functionality for distributed systems. Cluster management is very difficult, in the distributed system if the Zookeeper service, can make it easy for us to manage the heap group. Cluster management The most troublesome thing is node fault management, zookeeper can let the cluster choose a healthy node as the Master,master node will know the current cluster of each server health, once a node fails, Master will notify the other servers in the cluster to redistribute the compute tasks for the different nodes. Zookeeper not only can find fault, but also the fault of the server screening, see what the fault server is the fault, if the fault can be repaired, zookeeper can automatically repair or tell the system administrator the cause of the error to let the administrator quickly locate the problem, repair the fault of the node. People may have a question, master fault, what to do? Zookeeper considering this, zookeeper has an "election leader algorithm", master can be dynamically elected, when Master fails, Zookeeper can immediately select the new master to manage the cluster.

Next I want to talk about the characteristics of zookeeper:

1, zookeeper is a streamlined file system. It's a bit like Hadoop, but zookeeper this file system manages small files, and Hadoop manages huge files.

2, Zookeeper provides a wealth of "components", these components can achieve a lot of coordination of data structure and protocol operations. For example: Distributed queues, distributed locks, and "leader election" algorithms for a set of sibling nodes.

3, zookeeper is highly available, its own stability is quite good, distributed cluster can rely on the management of zookeeper cluster, the use of zookeeper rice flour Distributed system single point of failure.

4. The zookeeper adopts a loosely coupled interactive mode. This is most evident in the zookeeper offering distributed locking, zookeeper can be used as an appointment mechanism that allows the participating process to discover and interact with each other without knowing the other processes (or networks), and the parties involved do not even have to exist at the same time. As long as a message is left in zookeeper, after the process is finished, another process can read the message, decoupling the relationships between the nodes.

5, zookeeper for the cluster to provide a shared repository, the cluster can be centralized from here to read and write shared information, to avoid the sharing of each node programming, reduce the development of distributed systems difficult.

6, zookeeper design uses the observer's design pattern, zookeeper is responsible for storing and managing the data that everyone cares about, then accepting the observer's registration, once the state of these data changes, Zookeeper will be responsible for notifying those observers who have registered on zookeeper to respond accordingly, so as to achieve a similar master/slave management model for the urgent masses.

This shows that zookeeper is conducive to the development of distributed systems, it can make distributed systems more robust and efficient.

The application of the six-zookeeper of zookeeper series