Introduction to Quorum (NRW) algorithm mechanism

The Quorum mechanism is one of the many protocols and mechanisms used in the design of distributed systems to solve the problem of reliability and data consistency. We simply introduce it through a read-write model in a distributed system.

The reading and writing model in distributed system

Distributed system is composed of multiple nodes (referring to a server, storage device, etc.), because the network anomaly, downtime and other nodes can not guarantee the normal work, especially when the number of nodes is very large, the node of abnormal condition is almost certain. In order to ensure the normal operation of the system, to provide reliable services, distributed systems in the storage of data using multiple copies of data (note: The copy here is not only for backup, it can participate in the provision of system services) to ensure reliability, That is, reading data on one of the nodes fails, so you can turn to another node that has a copy of the same data back to the user. This process is transparent to the user. Then there will be inconsistencies in the data's copy data, for example: Once a user commits a change, the original saved copy is obviously inconsistent with the current data. The simplest way to solve this problem is to read the Write all, that is, after the user commits the modification, the system ensures that all copies of the stored data are fully updated, and then tells the user that the operation succeeds, while reading the data only needs to query one of the copy data to return to the user. This is a good solution in situations where the stored data is rarely modified, such as archiving historical data for later analysis. If you encounter frequently need to modify the situation, write operation time delay phenomenon is very obvious, coupled with concurrent or continuous implementation of the efficiency can be imagined. In essence, this is due to the unbalanced write and read load, read is easy, Write deep gauge pressure!

So is there a solution that does not need to update the complete part of the data, but also to ensure that the return to the user is a valid data solution? The quorum mechanism is an option.

Starting with the pigeon nest principle.

Pigeon Nest principle, also known as drawer principle, why from the drawer principle? One is familiar with this, and secondly it is similar to the quorum mechanism. Recalling the principle of the drawer, 2 drawers Each drawer can hold up to 2 apples, there are now 3 apples no matter how to put, one of the drawers there will be 2 apples. So we change the principle of the drawer, 2 drawers One put 2 red apples, the other put 2 green apples, we take out 3 apples, no matter how to get at least one is a red apple, this understanding is very simple. We look at red apples as updated valid data, green Apple as not updated invalid data. You can see that there is no need to update all the data (not all red apples) We can get valid data, of course we need to read multiple copies to complete (remove multiple apples). This is the prototype of the quorum mechanism, which essentially balances the load of write all to Read only.

Quorum mechanism

The above drawer theory is only the understanding of it, a simple generalization is that Quorum is a set, L arbitrarily take set s,r, s,r all exist intersection. Of course, this article does not intend to speak more about its mathematical definition of understanding, here is just to provide a message, do not understand and nothing to contact the front of the distributed read-write model can easily understand this.

Back to the beginning of the article, let's look at how to use the quorum mechanism to solve the read and write load balancing in the read-write model. In fact, the key is to update how many copies of the data, so that reading can always read the valid data? Recall our red apples, assuming that there is a total of N copies of data, where K has been updated, n-k not updated, then we arbitrarily read n-k+1 data when there must be at least one is the update of the k inside, that is, the intersection of Quorum, we only need to compare the n-k+1 version of the Read The highest data is returned to the user to get the latest updated data.

So what about writing a model? I also only need to complete the k copy of the update, you can tell the user operation completed without the need to Write all, of course, after the user completes the operation, the system will still slowly update the remaining copies, which is transparent to the user. As you can see, we have shifted some of the load on the write to read, read multiple copies, so that write is not too tired, the bad thing is to weaken the data consistency in the distributed system. As for the transfer of how much load is appropriate, this needs to be based on the specific requirements of the distributed system data consistency requirements. However, the CAP theory tells us that there is no perfect solution.

Redundancy Control algorithm

In distributed storage systems with redundant data, redundant data Objects store multiple copies between different machines. But at the same time multiple copies of a data object can only be used for reading or for writing. The algorithm can guarantee that multiple copies of the same data object will not be read and written by more than two Access objects.

Each copy of the data in the distributed system is given a vote. Each operation must be given a minimum number of votes (Vr) or minimum number of votes (Vw) to read or write. If a system has a V-ticket (meaning a data object has a v redundant copy), then this minimum read-write ticket must satisfy:

Vr + Vw > V
Vw > V/2

The first rule guarantees that a data is not read and written at the same time. When a write request comes in, it must obtain a license for the VW redundant copy. The remaining number is V-VW not enough VR, so no more reading requests come over. Similarly, when a read request has been licensed for a VR redundant copy, the write request is not licensed.
The second rule guarantees the serialization of data modification. A redundant copy of a piece of data cannot be modified by two write requests at the same time.

In distributed system, redundant data is the means of guaranteeing reliability, so the consistency maintenance of redundant data is very important. In general, a write operation must update all redundant data to be called a successful end. For example, a data on 5 devices have redundancy, because do not know the reading data will fall on which device, then a write operation, must be 5 devices are updated to complete, write operations to return.

For systems with frequent write operations, the bottleneck of this operation is very large. The quorum algorithm allows the write operation to return as soon as it finishes writing 3 units. The rest is done in slow synchronization within the system. Read operations, you need to read at least 3, to ensure that at least one can read the latest data.

Quorum's read-write minimum number of votes can be used as an adjustable parameter for the system in terms of read and write performance. The larger the number of votes, the larger the number of readers, the smaller the VR, the more expensive the system writes. Conversely, the cost of writing is small.

Https://qkldx.net/topic/599/quorum-nrw-%E7%AE%97%E6%B3%95%E6%9C%BA%E5%88%B6%E7%AE%80%E4%BB%8B

Introduction to Quorum (NRW) algorithm mechanism