Quorum mechanism in the reading and writing model of distributed system

The Quorum mechanism is one of the many protocols and mechanisms used in the design of distributed systems to solve the problem of reliability and data consistency. We simply introduce it through a read-write model in a distributed system. the reading and writing model in distributed system

Distributed system is composed of multiple nodes (referring to a server, storage device, etc.), because the network anomaly, downtime and other nodes can not guarantee the normal work, especially when the number of nodes is very large, the node of abnormal condition is almost certain. In order to ensure the normal operation of the system, to provide reliable services, distributed systems in the storage of data using multiple copies of data (note: The copy here is not only for backup, it can participate in the provision of system services) to ensure reliability, That is, reading data on one of the nodes fails, so you can turn to another node that has a copy of the same data back to the user. This process is transparent to the user. Then there will be inconsistencies in the data's copy data, for example: Once a user commits a change, the original saved copy is obviously inconsistent with the current data. The simplest way to solve this problem is to read the Write all, that is, after the user commits the modification, the system ensures that all copies of the stored data are fully updated, and then tells the user that the operation succeeds, while reading the data only needs to query one of the copy data to return to the user. This is a good solution in situations where the stored data is rarely modified, such as archiving historical data for later analysis. If you encounter frequently need to modify the situation, write operation time delay phenomenon is very obvious, coupled with concurrent or continuous implementation of the efficiency can be imagined. In essence, this is due to the unbalanced write and read load, read is easy, Write deep gauge pressure.

So is there a solution that does not need to update the complete part of the data, but also guarantees that the return to the user is a valid data solution. The quorum mechanism is an option.

from the principle of the drawer in elementary school

Why start with the drawer principle. One is familiar with this, and secondly it is similar to the quorum mechanism. Recalling the principle of the drawer, 2 drawers Each drawer can hold up to 2 apples, there are now 3 apples no matter how to put, one of the drawers there will be 2 apples. So we change the principle of the drawer, 2 drawers One put 2 red apples, the other put 2 green apples, we take out 3 apples, no matter how to get at least one is a red apple, this understanding is very simple. We look at red apples as updated valid data, green Apple as not updated invalid data. You can see that there is no need to update all the data (not all red apples) We can get valid data, of course we need to read multiple copies to complete (remove multiple apples). This is the prototype of the quorum mechanism, which essentially balances the load of write all to Read only.

Quorum mechanism

The apple drawer theory is just an understanding of it, citing references to the definition of quorum:

In a nutshell, the Quorum is a set, and the S,r of any collection in L, S,r, is the intersection. Of course, this article does not intend to speak more about its mathematical definition of understanding, here is just to provide a message, do not understand and nothing to contact the front of the distributed read-write model can easily understand this.

Back to the beginning of the article, let's look at how to use the quorum mechanism to solve the read and write load balancing in the read-write model. In fact, the key is to update how many copies of the data, so that reading can always read the valid data. Recall our red apples, assuming that there is a total of N copies of data, where K has been updated, n-k not updated, then we arbitrarily read n-k+1 data when there must be at least one is the update of the k inside, that is, the intersection of Quorum, we only need to compare the n-k+1 version of the Read The highest data is returned to the user to get the latest updated data.

So what about writing a model. I also only need to complete the k copy of the update, you can tell the user operation completed without the need to Write all, of course, after the user completes the operation, the system will still slowly update the remaining copies, which is transparent to the user. As you can see, we have shifted some of the load on the write to read, read multiple copies, so that write is not too tired, the bad thing is to weaken the data consistency in the distributed system. As for the transfer of how much load is appropriate, this needs to be based on the specific requirements of the distributed system data consistency requirements. However, the CAP theory tells us that there is no perfect solution.

Reference documents:

[1] Dahlia Malkhi,michael Reiter. Byzantine quorum systems[j], 1998

[2] David Peleg,avishai Wool. Crumbling walls:a class of practical and efficient Quorum systems[j], 1997