Advanced theory of Distributed systems-Paxos

Introduction

The theory foundation of distributed systems-conformance, 2PC and 3PC describes the issues of consistency, the need to reach a consensus, and the 2PC, 3PC model, where the Paxos protocol guarantees consistent resolution in the scenario of node downtime recovery, message sprawl or loss, and network differentiation. Is the most widely discussed consistency agreement.

Paxos agreement at the same time with its "obscures" known, the following combination of Paxos made simple, the part-time Parliament two papers, try to deduce through Paxos, learning and understanding Paxos protocol.

Basic Paxos

What is a consistency issue? In simple terms, the consistency problem is the question of how to reach a resolution between nodes that are independent, such as node outage, message unordered, and so on, as a protocol to resolve the consistency problem, the core of Paxos is how to determine and determine only one value in each node.

Maybe you'll be wondering just what a value can do, determining in the Paxos protocol that only one value is the basis for determining multivalued, and how to determine that multivalued will be covered in the second part of multi Paxos, which we focus on "how Paxos determines and determines only one value".

Similar to 2PC, Paxos first divides the node into two categories, initiating the proposal (proposal) on the side of proposer, the party involved in the resolution is acceptor. If there is only one proposer initiating the proposal, and the node is not down and the message is not dropped, then acceptor can determine a value by doing the following:

P1. A acceptor accepted the first proposal it received

Of course, the prerequisites for the above requirements are somewhat harsh, the node cannot be down, the message cannot be dropped, and only one proposer can initiate the proposal. We try to relax the condition, assuming that multiple proposer can initiate the proposal at the same time, how can we determine and determine only one value?

First of all, proposer and acceptor need to meet the following two conditions:

1. Each proposal initiated by proposer is identified by an ID, and the proposed composition becomes (ID, value)

2. Acceptor may accept (accept) more than one proposal, which is determined when the majority (majority, or quorum) acceptor accepts a proposal (chosen)

(Note: Note the difference between "accept" and "OK" above)

We agreed that the ID of the proposed proposal is larger than the one previously proposed, and that it is assumed that several proposals can be determined to determine and determine only one value acceptor to do the following:

P2. If a proposal with a value of V is determined, then only the proposed value of V is determined later

(Note: At first glance this condition is not very good to understand, remember the goal is "OK and determine only a value")

As a proposal is determined (chosen) must first be accepted by acceptor (accepted), in order to achieve P2, essentially acceptor need to do:

P2A. If a proposal with a value of V is determined, then acceptor only accept the offer with a value of V

satisfies the P2A namely satisfies P2 (P2A = P2).

Currently, when multiple proposer can initiate a proposal at the same time, satisfying P1, P2A is able to determine and determine only one value. What if we add the node downtime recovery and message packet loss considerations?

Assuming acceptor c is down for a period of time, the other acceptor has passed a resolution of V in the C outage period, but C is not known because of the outage; c if there is a proposer immediately after the resumption of the proposed value is not V, because the conditions P1,c will accept the proposal, This contradicts with the P2A. In order to avoid this situation, we are further constrained by the proposer:

p2b. If a proposal with a value of V is determined, then proposer only proposes a value of V

satisfies the p2b namely satisfies p2a (p2b = P2A = P2).

P2B constrains the behavior of the proposer after the proposal is determined (chosen), and we are more concerned about what the proposed proposer should do before it is determined:

p2c. For the proposal (N,V), acceptor in majority s, if there is acceptor most recent (that is, the maximum ID value) accepted by the proposed value of v', then v = v'is required; otherwise v can be any value

satisfies the p2c namely satisfies p2b (p2c = p2b = P2A = P2).

The description of p2c may feel confused, and we deepen our understanding through the example in the part-time Parliament:

Suppose there are a~e 5 acceptor,-said acceptor due to downtime and other reasons absent the resolution, X said Acceptor do not accept the proposal, O to accept the proposal, majority acceptor accept the proposal is confirmed, the above table corresponds to the resolution process is as follows:

1. The proposal for ID 2 was first proposed, according to which the proposed value of P2C could be any value, which is assumed to be a
2. Acceptor a/b/c/e did not accept any proposal in the previous resolution, so the proposed value of ID 5 can also be any value, which is assumed to be B
3. Acceptor b/d/e, where D had accepted the proposal for ID 2, according to P2C, the proposed value for the round ID 14 must be the same as the proposed value for ID 2, a
4. Acceptor a/c/d, where D had accepted the proposal of ID 2, C had accepted the proposal of ID 5, compared with ID 5 is larger than ID 2, according to P2C, the proposed value of the round ID 27 must be the same as the proposed value of 5, B; The round resolution was accepted by the majority acceptor, So the resolution was passed
5. Acceptor b/c/d,3 a acceptor prior to the proposal, in contrast to C, D has accepted the largest ID number of ID 27, the proposed value of the round ID 29 must be the same as the proposed value of ID 27, B

The above mentioned constraints can be summed up as 3 points, if the Proposer/acceptor meet the following 3 points, then in the case of a small number of nodes outage, network differentiation isolation, in the "OK and only determine a value" this thing can guarantee consistency (consistency):

• B1 (ß): ß each round of resolution has a unique ID identifier
• B2 (ß): If resolution B is accepted by the acceptor majority, the adoption of resolution B
• B3 (ß): For any proposal in ß B (n,v), acceptor majority if there is acceptor most recent (that is, the ID value of the largest) accepted the proposed value of V ', then v = V '; otherwise v can be any value

(Note: The Greek alphabet ß represents a set of multiple rounds of resolution, the letter B represents a round of resolution)

In addition, to ensure P2C, we have two constraints on acceptor:

1. Record the largest offer of the ID accepted, as proposer need to inquire about the information to determine the proposed value

2. In response to the proposal that the proposer of the proposal ID N had received the most ID, acceptor also ensured that (promise) no longer accepted the proposal that the ID was less than n

Proposer/acceptor complete a round of resolutions can be summarized as the prepare and accept two stages, shown as follows:

There is also a question to consider, if proposer a initiated the proposal for ID N, Proposer B also initiated the proposal for ID n+1 before the proposal was completed, Proposer C before the N+1 proposal was completed, and the proposal to launch the ID n+2 ... This does not complete the resolution, the formation of a live lock (livelock), although this does not affect consistency, but we generally do not want to let such a situation happen. The solution is to select a leader from proposer and unify the proposal from proposer leader to avoid a live lock.

Finally, we introduce a new role: Learner,learner is attached to acceptor, and is used to acquire the adopted resolution. The above resolution process only requires acceptor majority participation, and we want to try to be consistent with all acceptor. If some acceptor due to downtime and other reasons not known to have passed the resolution, downtime after the recovery can be learner by the use of pull in the way from other acceptor acquisition.

Multi Paxos

With the above steps the distributed system has been able to determine a value, "What is the use of only one value?" This will not solve the problem I am facing. "You may have such doubts in your mind.

In fact, the process of "Determining a value" can be used to determine the series values with full order (total order), which can be applied to many scenarios such as database copy storage. We refer to the process of "determining a value" as an instance (instance), which consists of Proposer/acceptor/learner, which illustrates the example of a a/b/c on three machines:

The different ordinal instances do not affect each other, the A/B/C three machines input the same, the process is essentially equivalent to executing the same sequence of state machine instructions, thus will get consistent results.

Proposer leader in multi Paxos also helps to improve performance, the normal unified by leader to initiate the proposal, you can save prepare step (leader do not inquire acceptor have received the largest proposal of the ID, Only the leader proposal does not require acceptor to promise) until leader downtime, network drops, etc. occur.

Summary

The above introduces the Paxos process, how to build multi Paxos through the state mechanism on the basis of basic Paxos. Paxos protocol comparison "obscures", but read a few times the paper can generally understand its connotation, more difficult is how to apply the Paxos to engineering practice.

Backstage development students realize and open source a set of multi-machine state copy class library based on Paxos protocol Phxpaxos,phxpaxos is used to extend the stand-alone service to multi-machine, which is verified by the system and has many considerations on the consistency guarantee, performance and so on.

--

Some of the concepts mentioned in this article include consistency (consistency), consistency system model, majority (quorum), full order relationship (total order), etc., which are described in the following articles:)

Theory Foundation of distributed systems-conformance, 2PC and 3PC

Theory Foundation of distributed systems-time, clock and sequence of events

The theory base of distributed systems-CAP

--

Advanced theory of Distributed systems-Paxos