Theory of distributed systems-from abandonment to getting started

With the increase of the number of bearer users and the need for disaster recovery, more and more Internet background system from the single-machine mode switch to distributed cluster. Looking back on what I've been doing for five years, there's also a shift.

The first two years of graduation is responsible for maintaining the operation of the blade in the business, in the machine room to pull the single board day is my lost youth . A cold standby is formed between the devices through VCs, but even with dual-computer software protection, outages, network drops, and so on, can still be impacted by the business. In such a system architecture to guarantee SLAs, it is sometimes necessary to delve into the Linux system kernel or hardware level to analyze the cause of machine restarts.

Next is responsible for the maintenance of the distributed cluster of business, compared to the previous work, this stage is mainly concerned about the point is not a single node anomaly, more is the overall stability and robustness of the system. In the face of the complex system, at the beginning of this feeling:

Before the large and complex distributed system, where should we start to raise awareness and understanding and enhance professionalism? Many distributed system related papers and materials can be found on the internet, but what is the main meaning to be summed up?

Based on the experience of these years, the core of the distributed system is to solve a problem: how to reach consensus among different nodes .

Seemingly simple problems arise from the complexity of network drops, node downtime and other scenarios, resulting in a number of concepts, protocols and theories. In order to explore the most possible solution to the consensus problem, there is the FLP, cap boundary theory; To resolve the problem within specific conditions and scope, there are consistency agreements Paxos, Raft, Zab and viewstamped Replication; to build these protocols, a majority, Leader concepts and methods such as elections, leases, and logic clocks.

2016 I read a number of representative papers and blog posts in the field of distributed systems, focusing on the question of how different nodes can reach consensus, adding their own understanding and understanding to the following 7 summaries:

Consistency, 2PC and 3PC
Elections, majorities and leases
Time, clock, and sequence of events
CAP
Paxos
Raft, Zab
Paxos Variants and optimization

The idea and writing technique article is a hard process, on the one hand to read a lot of information and to translate into their own understanding, to find as far as possible the concept and angle of merely, on the one hand to brain tissue language to make the intended reader easy to understand.

But it is also an interesting process, the knowledge has been smoothed over the original vague concepts become clear, the writing process to think of some interesting content I will also be inserted into the article, sometimes I think of some small clever to amuse:)

I hope this can be a systematic introduction of distributed theory of Chinese data add a brick, add a tile.

Theory of distributed systems-from abandonment to getting started