Network Programming (vii): CAP principle Derivation and application

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M02/78/DF/wKioL1aElX7BVw0gAAFx3p8odVw169.jpg "title=" 0641de9fd6362b6b9cb78649942ddca0_b.jpg "alt=" wkiol1aelx7bvw0gaafx3p8odvw169.jpg "/>

in theoretical computer science, the cap theorem (Cap theorem), also known as the Brewer's theorem (Brewer's theorem), points out that it is impossible for a distributed computing system to meet the following three points simultaneously:

• Consistency (consistency) (equivalent to all nodes accessing the same copy of the latest data)

• Availability (availability) (High availability for data updates)

• Network partition tolerance (Partition tolerance) (in effect, partitions are equivalent to the time-limit requirements for communication.) If the system cannot achieve data consistency within the time frame, it means that the partition has occurred and that a choice must be made between C and a for the current operation. ）

According to the theorem, the distributed system can meet only two of the three items, but it is impossible to satisfy all three items.

The simplest way to understand the cap theory is to imagine two nodes on each side of the partition. Allowing at least one node to update the state results in inconsistent data, i.e. the loss of C nature. If the node on the partition side is set to not be available for data consistency, the A property is lost. Unless two nodes can communicate with each other, both C and a are guaranteed, which can lead to the loss of P-Properties.

This theorem originates from the computer scientist Eric Brour of the University of California, Berkeley (University of California, Berkeley) in the 2000 Distributed Computing Principles Workshop (Symposium on Principles of A conjecture proposed by distributed Computing (PODC). In 2002, Seiss Gilbert and South Hill Linch of the Massachusetts Institute of MIT (MIT) published a proof of Brewer's conjecture, making it a theorem. Gilbert and Lynch proved the cap theorem Bibrur to some extent more narrowly conceived. The theorem discusses the processing scheme when two conflicting requests reach two different distributed nodes that are not connected to each other. from :wikipedia.org

When I finished reading Wikipedia for the first time, the explanation was indefinitely, and now I understand that the author of Wikipedia's article is Abas.

example derivation of the CAP principle

Let's take the database as an example to discuss the CAP principle.

Single Instance

First, let's discuss one of the simplest cases: "Single db instance". This is also the most common small site, personal blog, Small forum architecture.

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/78/E0/wKiom1aEkp7CMsGEAABlzpiLYQQ220.png "class=" origin _image zh-lightbox-thumb "width=" 698 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "Ebc7d2819bde9f31a64c600e52f55e82_b.png" alt= "Wkiom1aekp7cmsgeaablzpilyqq220.png"/>

We can easily analyze, because of single instance, so there is no "network partition", "inconsistent", but a single point of failure will cause the entire database to be paralyzed, so availability is not guaranteed.

This is the cap theorem, which guarantees "C" and "P", and discards "A".

All standalone systems fall into this category, such as MySQL, memcached, and Redis.

sharding

Sharding can be translated into shards.

In order to improve usability, we often apply some hashing algorithms to the client in the actual production environment for data shard storage, as shown in:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/78/DF/wKioL1aEk1_zxkK5AABwIsySNug561.png "class=" origin _image zh-lightbox-thumb "width=" 778 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "E4aa992e9783a0b0d64875fee506105f_b.png" alt= "Wkiol1aek1_zxkk5aabwisysnug561.png"/>

Data consistency is guaranteed because the data is fragmented and stored in each database.

Because the databases do not communicate with each other and do not rely on each other, the tolerance of partitioning is still not compromised.

What about usability? Most of the time someone will take a direct shot at the head, here we use the mathematical way to solve the problem.

Assuming that the cluster has two servers, the data is evenly distributed, the probability of our DB instance downtime is p. The availability of this cluster using hashing for data fragmentation is:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/78/E1/wKiom1aEk2zSEtRLAAAdsRRiQAQ709.png "class=" origin _image zh-lightbox-thumb "width=" 896 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "B7979b2c044f3bb8f998629693059c2c_r.png" alt= "Wkiom1aek2zsetrlaaadsrriqaq709.png"/>

Even if the data is evenly distributed or the number of clusters increases, the result is the same: "Cluster availability is still P".

That we have been tossing for a while, cap and single-machine unexpectedly is the same, for a ball ah? In this case, the CAP indicators do not improve, but the benefits are:

1. A single server outage can only result in service degradation;

2. The cluster has the possibility of expanding and shrinking, which is called scalability.

This distributed approach is often used to:

• distributed memcached, Redis

• traditional database sharding

• Span style= "FONT-SIZE:12PX;" >bigtable (Columnstore database)

• hypertable (Columnstore database)

• hbase (Columnstore database)

• mongodb (document database)

• terrastore (document database)

• redis (KV database)

• scalaris (KV database)

• memcachedb (KV database)

• berkeley db (KV database)

multiple copy writes

What should we do if we want to ensure higher availability? We have the following approach:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/78/E1/wKiom1aEk6aRf71ZAAB190OEjoM587.png "class=" origin _image zh-lightbox-thumb "width=" 842 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "Faa86dbe7cb3eb930b99bbabac7374ab_b.png" alt= "Wkiom1aek6arf71zaab190oejom587.png"/>

Client multi-Copy writing is the client writes to multiple databases at the time of writing the database, and is considered successful after all two writes are successful. This approach greatly improves the availability of reads because there are multiple copies of the data. However, because you write more, all instances of the replica must be available to succeed. As a result, the usability of the write has declined.

Assuming that the failure rate of a single-machine database is P (p<1.0), the availability of a stand-alone database is 1-p. The summary is:

• In the write scenario, consistency (C) and partition tolerance (P) have not changed, and availability (A) has declined, from 1-p down to 1-2P-P2

• In the read scenario, consistency (C) and partition tolerance (P) remain unchanged, and availability (A) rises, rising from 1-p to 1-P2

We can further conclude by:

The "client multi-copy write" approach is ideal for improving usability in scenarios where "read and write Less".

There are "variants" in this way to improve the poor usability of writes, such as:

• A successful partial copy of the write succeeds and the remaining copy is written with no guaranteed results. The result of this is to sacrifice some consistency (C) in exchange for an increase in availability (A).

Generally speaking, this kind of way belongs to the category of "sharding" in the broad sense, and there is a bigger problem to remove the above shortcomings:

Assuming that the number of replicas is n,client to write to a single instance, the time-consuming T, multi-copy write time is n*t, when n > 1 will affect the client's write performance exponentially.

Clustering

In order to solve the problem of complex client write slow call, we introduced the cluster scheme, namely clustering. clustering and sharding are compared as follows:

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/78/DF/wKioL1aElACgdgRxAACk4jdJ4oA689.png "class=" origin _image zh-lightbox-thumb "width=" 930 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "1813ef3a81e938baec57b5fe47657734_b.png" alt= "Wkiol1aelacgdgrxaack4jdj4oa689.png"/>

Multi-Copy mode:

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M00/78/DF/wKioL1aElEaAQieSAACbIW1DyOU588.png "class=" origin _image zh-lightbox-thumb "width=" 1076 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "B948f04dd22849b3f839e19a3e8af771_b.png" alt= "Wkiol1aeleaaqiesaacbiw1dyou588.png"/>

We can see that consistency (C) is still guaranteed because multiple copies of the copy are successfully returned. However, write availability (A) and partition tolerance (P) are degraded relative to the single machine. In exchange for:

1. The more simple API, the client does not pay attention to "write" problem;

2. High Availability (HA) for read operations.

As the above scenario is strongly consistent (C), this scenario is common in the financial system, a typical representation of which is:

• ZooKeeper (KV database)

• Vertica (Columnstore database)

• Aster data (relational database)

• Greenplum (relational database)

Similar to the scenario we adopt in "Sharding", the database on the production environment line is also often used to abandon certain consistency (C) to improve availability (A) and partition tolerance (P).

Such as:

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M02/78/DF/wKioL1aElMDjRw30AACb_o0HwOQ414.png "class=" origin _image zh-lightbox-thumb "width=" 1056 "style=" border:0px;height:auto;vertical-align:middle;margin:20px auto 30px; " Title= "41d7018aa8fa5b3b0a7f032b38d0234d_b.png" alt= "Wkiol1aelmdjrw30aacb_o0hwoq414.png"/>

As you can see, because the needs of most Internet companies do not require strong consistency (C), achieving higher availability (A) and partition tolerance (P) is the core idea of most NoSQL databases on the market by abandoning consistency.

In Amazon's well-known distributed database Dynamo, it is a similar approach: "3 copies, 2 successful after the return of success, the remaining 1 copies of the subsequent synchronization", we call this mode "final consistency"

Typical representatives in this area are:

• Dynamo (KV database)

• Voldemort (KV database)

• Tokyo Cabinet (KV database)

• KAI (KV database)

• Cassandra (Columnstore database)

• CouchDB (document-type database)

• SimpleDB (document-type database)

• Riak (document-type database)

• Moosefs (class GFs Distributed File System)

Basically, there are several scenarios where all distributed systems can be covered. We can see that the system design is like a poor family quilt, cover the head and left foot on the right foot, cover the head and right foot on the left foot ... Even if you have the money, you will not be able to satisfy the cap 100%.

So the question is, is it possible to raise the cap at the same time? The answer is: Sure, ofcourse.

We can reach the cap and improve it by using more reliable servers and better network equipment.

For example, suppose a bank has just started using a 2w-dollar Dell Server, because of the particularity of the banking business, the consistency (P) is the primary guarantee (no bank is willing to see, and at the same time the ATM to take 100 of the account, all succeeded) so this server may be a single point, availability (a) Very bad. The banks are rich, the servers are replaced by IBM machines, and the CPUs are two-way hot backups. Availability (A) is naturally not 2w comparable to Dell.

But that's the word:

even if you're rich, you can't find a quilt that covers both your legs and your head. .

BASE & ACID

We know that the transaction of the database has an acid guarantee:

Transaction mechanisms ensure data consistency.

A transaction should have 4 properties: atomicity, consistency, isolation, persistence. These four properties are often called acid properties.

• Atomicity (atomicity). A transaction is an inseparable unit of work, and the operations included in the transaction are either done or not.

• Consistency (consistency). The transaction must be to change the database from one consistency state to another. Consistency is closely related to atomicity.

• Isolation (Isolation). Execution of one transaction cannot be disturbed by other transactions. That is, the operations within a transaction and the data used are isolated from other transactions that are concurrent, and cannot interfere with each other.

• Persistence (durability). Persistence, also known as permanence (permanence), refers to the fact that once a transaction is committed, its changes to the data in the database should be permanent. The next operation or failure should not have any effect on it.

Later, with the rise of NoSQL, the technology community proposed the concept of base:

• Basically availble--Basic available

  Support partition failure (sharding fragmentation database), the problem service is degraded only (partially unavailable).

• Soft-state--Soft State/flexible

  The transaction "Soft state" can be understood as "no connection", while "hard state" is "connection oriented". A soft state is one that can be unsynchronized for some time, asynchronously.

• Eventual consistency--final consistency

  The final data is consistent and not always consistent.

Together it is base.

It is interesting to note that acid is the meaning of acid in English, and base has the meaning of alkali.

Copyright belongs to the author

commercial reprint please contact the author for authorization non-commercial reprint please specify the source.
link http:// zhuanlan.zhihu.com/auxten/20315482

==================================

"want to learn more wonderful content welcome attention"

contact reboot-have more technology to share, Exchange please Dabigatran: 238757010

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M02/78/E1/wKiom1aElRORvwMSAAEHk3D-4sI359.jpg "title=" Erweima.jpg "alt=" Wkiom1aelrorvwmsaaehk3d-4si359.jpg "/>

This article is from the "Reboot DevOps Development" blog, please be sure to keep this source http://opsdev.blog.51cto.com/2180875/1730262

Network Programming (vii): CAP principle Derivation and application