Into the cloud-chubby

Chubby is simply a distributed lock service, where thousands of the client's chubby can "lock" or "unlock" a resource, and collaborative work within systems such as bigtable and MapReduce often uses chubby, In implementation is the use of the well-known scientists Leslie Lamport Paxos algorithm, through the creation of operational files to achieve "lock."

In implementing the mechanism, chubby itself is a distributed file system, and provides mechanisms to enable the client to create files on the chubby service and perform basic operations on some files. Chubby through the file to achieve the "lock" function. File is the "lock" in the chubby, the creation of the file is actually "lock" operation, preemption to the "lock" is the server successfully created the file. When a user opens, closes, and reads a file, it acquires a shared or exclusive lock, and then sends updates to the user via a communication mechanism.

On the architecture, the chubby cluster typically consists of 5 machines, each with a replica (replica), one of which replica is selected as master node, replica in structure and capability, Replica uses the Paxos protocol to keep the log consistent, and replica can be offline and then back online. Once you are back online, you need to maintain consistent data with other nodes. The client side is accessed using the Chubby library.

Main advantages

Why not implement a protocol like the Paxos algorithm directly to solve the consistency problem, but to solve it through a lock service? This has the following five main benefits:

1. Most developers do not take this consistency into account when they start developing services, so they do not use a consistency protocol at first. It is only when the service matures that the problem is taken seriously. The use of lock service can make it possible to solve the problem of consistency by adding simple statements while maintaining the original program architecture and communication mechanism.

2. In many cases, it is not just the election of a master, but also the need to tell the master's address to others or to save a message, when using the files in chubby, not only to provide the lock function, but also to record useful information in the file ( such as Master's address). So, many developers use chubby to save metadata and configuration.

3. A lock based development interface is more easily familiar to developers. Not all developers understand the consistency protocol, but most people should have used locks.

4. A consistency agreement generally requires several replicas to ensure high availability, in which case the Paxos algorithm is the most obvious example, and the use of chubby, even if only one client can be used.

5. It can be seen that the lock service is in this form because chubby not only wants to solve the problem of consistency, but also provides more useful functionality. In fact, Google has a lot of developers who use chubby as the name service, and it works very well.

Related Products

As with the GFS, MapReduce and BigTable described earlier, there is also a class chubby implementation in the Hadoop series called Zookeeper. In terms of implementation, zookeeper is based on a set of independently designed and optimized two-phase commit protocols and has been successfully applied to HBase, Yahoo! message Broker, Fetch Service of Yahoo! Crawler system.