Concepts in distributed systems--first consistency protocol, conformance model, Byzantine problem, lease

1, conformance Agreement

Two-phase commit agreement and Raft agreement, Paxos Agreement

① Two-phase commit agreement

In a distributed system, each node, while aware of its own operation, succeeds or fails, does not know the success or failure of the operation of the other node. When a transaction spans multiple nodes, in order to maintain the acid nature of the transaction, it is necessary to introduce a component that acts as a coordinator to unify the operation results of all nodes (called contributors ) and ultimately indicate whether they want to commit the results of the operation ( such as writing updated data to disk, etc.). Therefore, the two-phase submission algorithm can be summarized as follows: Participants will notify the coordinator of the success or failure of the operation, and then by the coordinator based on the feedback of all participants to determine whether the participants to commit or abort the operation.

Therefore, the system consists of two types of nodes, one is the coordinator, the other is the participants, the implementation of the Protocol consists of two stages:

Specific reference: Two phase commit: Wikipedia

The two-phase protocol is blocked, and the node waits for the other person to answer the message, and it cannot do anything else and the resources it holds are not released. It is primarily used to ensure atomicity of operations across multiple nodes-either all operations or none, and like the raft protocol, such as the one used to guarantee the consistency of operations, where each node performs the same operation .

Example of a two-phase protocol: Two-phase commit protocol

②raft Protocol and Paxos Protocol

The basic functions of raft and Paxos in distributed applications are similar, but Paxos is difficult to understand, relatively raft algorithm is simpler. There is a classic paper on the Raft protocol:

In Search of an understandable Consensus algorithm (Extended Version)

Translation Address reference: Finding an easy-to-understand consistency algorithm (extended version)

Another article explains in detail the relevant implementation of the RAFT algorithm: the 31st reference of the raft paper.

Here are just a few things to look at in the course of a paper:

Why does "most rules" guarantee that for a given term, only one candidate will eventually win the election to become a leader?
In raft, for a given term number, each server only casts a maximum of one ticket for that term number on a first-come-first-served basis, and if a candidate sends a request to vote RPC with a term number of more than half of the server's consent, Then the candidate becomes leader.
It is because each server only has a maximum of one vote per term and more than half of the votes to become a leader, so in a given term of vote, only one candidate will eventually become leader.

About the leader election steps in the Raft Agreement reference: Raft Series Article II: Leader election

2, three kinds of consistency models common in distributed system

① Strong consistency: When the update operation succeeds on a replica, all of the read operations must be able to obtain the latest data. For a single copy, read and write operations are performed on the same data, which is easy to ensure consistency, and for multi-replica data, a distributed protocol such as the 2PC protocol is used.

② Weak consistency: When you update a data, it takes a while for the user to read the latest data.

③ Final Consistency: It is a special form of weak consistency. It does not guarantee that when a data x is updated, the new data will be visible at all subsequent operations on X, but will require a time fragment to be guaranteed after the time fragment has elapsed . Within this time fragment, the data may be inconsistent and the fragment is called "Inconsistent window".

Reference: Data consistency model

3, Byzantine general question

In distributed theory, it is often seen that "in non-Byzantine error situations, the algorithm is effective ..." Or say "... Can tolerate non-Byzantine failure ". So what is the Byzantine general question?

In distributed computing , different computers try to reach consensus through exchanging information, but sometimes, the system coordinates the computer (Coordinator/commander) or the member computer (Member/ lieutanent) may affect eventual system consistency due to system errors and exchange of incorrect messages. Reference: Wikipedia: Byzantine general question

That is, the wrong message is passed between different machines in the system, which is the Byzantine issue. This is different from "network segmentation, Machine crashes ...". For example, the raft protocol does not tolerate Byzantine problems, but it can guarantee the correctness of its operation in the case of non-Byzantine errors, such as network delay, partitioning, packet loss, redundancy, and disorderly ordering.

4, Lease, Heartbeat pack (Heartbeat)

① Lease

Here are the main examples of what leases can be used for?

A, for fault detection. This is similar to the effect of heartbeat packets sent between master and Slaver in zookeeper. In ZK, the heartbeat packet is exchanged between master and slaver to detect if they are still alive. A specific example of reference: fault detection

b, maintaining cache consistency maintains the consistency of the cache, the first approach is polling: each time the data is read to ask whether the server data is up-to-date, if not, let the server transfer the new data before reading the new data. The second method is callback: the server records which clients read the data, when the server changes the data to notify the logged clients first, the last read data has been invalidated. These two methods have some drawbacks, reference: Lease mechanism Introduction

Therefore, a lease mechanism can be introduced. During the lease term, you can ensure that the data cached by the client is up-to-date. When the lease expires, the client needs to re-ask the server for data and renew it.

Definition of Lease: A lease is an agreement that gives the holder specific powers within a certain period of time . Each lease has a time limit, which guarantees that the lease mechanism will tolerate machine failure and network segmentation.

Lease mechanism An example of optimizing background database access: Using the lease mechanism to solve the problem of updating cached data

Concepts in distributed systems--first consistency protocol, conformance model, Byzantine problem, lease