Full Order, the essence of distributed consistency

A Brief History of consensus _ 2 PC and transaction commit

Time clocks and the ordering of events in a distributed system -- Leslie Lamport

 

Partial and full order

Lamport's "time, clocks and the ordering of events in a distributed system" (1978)

1. How to define partial order? How can we convert a forward order to a full order?

2. When explaining the full-order relationship, it involves the state machine problem of the distributed system, and proposes that the full-order problem can be used to solve the consistency problem of the distributed system.
I soon realized that the algorithm that defines the full-order relationship of events can be used to implement any distributed system.
A distributed system can be described as a special serial state machine with multiple network-connected processors. if you can sort all input requests, You can implement any state machine consisting of a network-connected processor. Therefore, you can implement any distributed system.
To demonstrate this, the paper uses a simple distributed system instance-distributed mutex algorithm as an example.

 

Partial Order and full order. Full Order is a special partial order.

Set a to a non-empty set, and P to a relation on A. If the relation P is self-inverse, called, and transmitted, P is the partial order relation on set.

In set a, if there is an ARB or bra for any A, B, and a, that is, every pair of elements in a satisfies the relationship R, then, the partial r on set a is in full order.

In full order, any two elements in the set are ordered and linear.
Partial Order: only some elements have sequential relationships, non-linear relationships, such as concurrency relationships in distributed systems.
For example, in a, V2 and V3 are concurrent and wireless, so a indicates partial order.
For B, you only need to add a line between V2 and V3 to change to a full-order relationship.

Partial Order, logical clock

The order problem is actually the definition of the "happen before" relationship. How can we define the occurrence of a before B?
Simple adoptionPhysical clock (absolute frame of reference)Yes? Yes, provided that there is no error in the physical clock in different places.
This is almost impossible, so physical clock synchronization is required, which is also a research direction... the core technology of spanner recently released by Google includes satellite clock synchronization between global datacenter.

Since it is difficult to use an absolute physical clock, the author thinks of the special theory of relativity, that is, through strictCausal relationshipTo determine the order
For the reference system, the selection of different reference systems may have different sequence judgments, but for the strict causal relationship, no matter what the reference system does not change

Special relativityIt tells us that events in time and space do not have a consistent full-order relationship. Different observers may have different opinions on the first occurrence of two events. only when event E2 is caused by event E1, there is a sequential relationship between event E1 and event E2.

There is a clear causal relationship between events in the process, for example, there is a fixed order for process P, P1, P2, P3, P4.
However, for events between processes, if the two processes are not associated or communicate with each other, the order cannot be determined.
Only when two processes communicate with each other, a causal relationship is generated to determine the order.
For example, if P1 sends a message to Q, while Q2 receives a message event from P, there is a clear causal relationship between P1 and Q2. Therefore, P1 "happen before" Q2, that is, P1-> Q2

For events that do not have a clear causal relationship, they cannot judge their order. They can only be considered as "concurrent"
For example, for P1 and Q1, you cannot determine which occurs first.

Partial Order '->' Definition

The minimum relationship that meets the following three conditions:
(1) If a and B are events in the same process and a occurs before B, A-> B
(2) If a represents the message sending event of a process and B represents the Receiving Event of the same message in another process, a-> B
(3) If a-> B and B-> C, A-> C (pass-through)

IntroductionLogical clock)To indicate the relative relationship. The so-called logic is not like the specific time of the physical clock record, but this is to express the relative relationship (that is, the causal relationship) between events by means of incremental numbers)

Logical clockDefinition

Clock condition. For any event A, B: If a-> B, then C (a) <C (B), otherwise, because it may be concurrent

C1. if both A and B are events in process PI and A is before B, then Ci (a) <Ci (B)

C2. if A is a message sending event in process PI and B is a message receiving event in PJ of another process, then Ci (a) <cj (B)

In the right figure, if the dotted line represents the ticks of the logical clock (the counting unit of the logical clock), there is at least one tick between events in the process, and the message line between processes spans at least one tick.

To ensure that conditions c1 and c2 are met, the process only needs to comply with the followingImplementationRules:

Ir1. each process PI increases the CI value between any two consecutive events.

Ir2. if event a represents the event in which the process PI sends the message m, the timestamp TM of the message m is CI (). after receiving the message m, the process PJ sets the CJ value so that it is greater than or equal to its current value and greater than TM.

Ordering the events totally, in full order

What are the issues to be solved in full order? How can I sort the concurrent statements?
In fact, there is no absolute standard. In this article, we use any full-order relationship pI <PJ between processes to determine the order. As for how to define the full order between processes, we can customize them.
Of course, you can use other standards to produce full order
The conclusion is that the partial order is uniquely determined by the events in the system, and different clock condition can produce different full order relations.

Full-order relationship "=>"Definition
Assume that A is an event in process PI and B is an event in process PJ. if and only when the following conditions are met:
(1) CI (a) <cj (B );
(2) CI (A) = CJ (B) and pI <PJ,
Then we think "A => B"

 

The value of the Full-order problem to solve the distributed consistency problem

The following conditions must be met,
(1) A process that has obtained resource authorization must be released before the resource is assigned to another process;

(2) resource requests must be authorized in the order of requests;

(3) After all processes that obtain resource authorization finally release the resource, all resource requests must have been authorized.

In fact, the key to this problem lies in (2) how to determine the order of requests in a distributed environment?
Is it feasible to use a central scheduling process in the order in which resource requests are received? (Security is not considered, that is, spof issues)
This paper considers that the request is not feasible because the request is in the order of request occurrence. Due to network problems, the request receiving order may be completely different from the order of occurrence.

Therefore, we need to use the clock system discussed above and provide a full-order relationship for all resource requests and release events.
First, create a clock system and add a logical clock (timestamp) to all application and release events)
Both application and release require all processes to send a message with a timestamp.
When receiving an application, all processes need to reply to a message with a timestamp.
When a process has received messages from all processes, you can use the full-order definition above to sort all events in the full-order "=>"
If the request of this process is at the top of the list, the resource can be occupied; otherwise, the request must be released by other processes.
Here, the message must be received in the sending order. If a message with a timestamp> TM is received from all processes, it can be proved that my request is the first to occur.

First, let us assume that for any two processes PI and PJ, the messages transmitted between them are received in the sending order, and all the messages will eventually be received.
Each process maintains a request queue that is invisible to all other processes. Assume that the request queue has only one message T0: P0 resource request at the initial time. P0 indicates the process that obtains resource authorization at the initial time. t0 is smaller than the initial value of any clock.
1. to request this resource, process PI sends a TM: PI resource request message to all other processes and puts the message in its own request queue. Here TM represents the message timestamp.

2. When the process PJ receives the TM: PI resource request message, it puts it in its own request queue and sends a timestamp confirmation message to pi. (Note: If PJ has sent a message with a timestamp greater than TM, it can be left blank)

3. When the resource is released, the process PI deletes all TM: PI resource requests from its own message queue and sends a time stamp PI resource release message to all other processes.

4. When the process PJ receives the PI resource release message, it deletes all TM: PI resource requests from its own message queue.

5. When both of the following conditions are met, the resource is allocated to the process Pi:

A) after sorting by the "=>" relationship, Tm: PI resource requests are placed at the top of its request queue.

B) I have received a timestamp> TM message from all other processes.

The problem with this algorithm is that the process fail is not allowed and there are ideal assumptions about the network conditions. Otherwise, full order cannot be found.
Therefore, the theoretical and enlightening significance is greater than the practical significance.
This idea inspired paxos, vector clock, and other related algorithms.
The paxos algorithm provides better answers to problems with strong consistency in unstable environments.
For the high availability problem of high concurrency, the vector clock technology provides a partial order solution based on this paper.