Problems to be solved by service design

A few days ago chatting with colleagues, colleagues said:

"The Service of the business (the underlying technology on our infrastructure side) requires a technical solution to three issues: distributed, communication, and storage. ”

I recalled the time I had been doing business before, and I felt that it was almost like a "service governance". Think of the "service design to solve the problem" This topic can be written before static a lot of articles to do a summary. Make a summary today.

Distributed

The common problem to be solved is the consistency of distributed transactions.

Rigid transactions and flexible transactions

Rigid transactions: Transactions that strictly follow the acid principle (atomicity, consistency, isolation, persistence). It basically refers to a local database transaction. Under the CAP principle, the transaction under distributed is not a rigid transaction.

Flexible transactions: Transactions that follow the CAP theory or its variant base theory. Distributed transactions are basically flexible transactions.

Because a rigid transaction is basically equivalent to a local database transaction, a flexible transaction is basically equivalent to a distributed transaction. Just one is distinguished by the rigor of the transaction, and one is differentiated by the usage scenario. Therefore, in addition to making comparisons, few directly mention the concept of rigid and flexible transactions.

Theory of distributed transactions

Distributed transactions: In a distributed environment, each operation step is not on the same machine, it needs to ensure that all actions have a uniform result of a set of operations.

The CAP principle (remember that in the previous blog): In a distributed environment, data consistency, service availability, partition fault tolerance can only meet the two.

Data Consistency (consistency): Here the consistency is strong consistency, also known as linear consistency. That is, a write operation succeeds, and the read must be the new data after the write operation. The write operation failed to read out the old data before the write operation.

Service Availability (availability): All operations can be responded to within a certain amount of time.

Partition fault tolerance (partition-tolerance): In the network partition environment, the partitioned node can still provide services externally.

Select description

AP-separated nodes that serve but do not communicate with each other will result in inconsistent state, i.e. not meeting C

CP Network Partition in the case of a C, the request can only wait, that is, does not meet a

CA in a certain period of time to achieve the same node state, requires that the network partition cannot occur, you can not meet the P

Base theory: This is the result of a tradeoff based on the CAP theory. The core idea is that even if strong consistency cannot be achieved, some technical means can be used to achieve eventual consistency. Base is the abbreviation for basically Available (basic available), Soft State (soft), eventually consistent (final consistency).

Distributed Transaction Conformance Implementation scheme

In order to solve the problem of distributed consistency, many classical protocols and algorithms are summed up in the process of tradeoff between performance and data consistency. The more famous are: 2PC, 3PC, TCC, Paxos, Raft, Zab, ISR. In addition to these, the most used in the industry is actually based on MQ implementation.

2PC two Phase commit: The two-phase commit is generally said to be based on the XA protocol. In addition, the JTA protocol is also more common.

XA is a distributed transaction protocol. It is broadly divided into two parts: the transaction manager and the local resource manager. Local resource managers are often implemented by databases, such as Oracle, DB2, and XA interfaces. MySQL's support for XA is not very good. The transaction manager, as the global dispatcher, is responsible for the submission and rollback of each local resource.

The advantages of the two-phase commit are: simple principle and convenient implementation. The disadvantage is synchronous blocking, single point problem, inconsistent data.

3PC (three Phrase commit) three-phase commit: Divided into Cancommit, Precommit, do commit three stages. is to divide the two-phase commit phase 1 into two, pre-commit when a participant returns no or a timeout breaks the transaction.

The advantage of a three-phase commit is to reduce the scope of participant blocking and to continue to agree after a single point of failure. The disadvantage is that because of the precommit phase, if a network partition occurs at this stage, the coordinator cannot communicate with the participants properly, the participants will still commit in kind, resulting in inconsistent data.

TCC (Try-confirm-cancel)

Try: Complete all checks, reserve the required resources

Confirm: Execute the business using a resource reserved in the try phase and retry if an exception occurs

Cancel: Release the try phase Reserve resource

TCC is capable of locking individual resources in a distributed transaction, committing and releasing them separately. It is suitable for scenes with strict consistency, short execution time and high real-time requirement.

Paxos algorithm: Previously saw "from Paxos to Zookeeper" that book, did not see how to understand. Implementation is more complex, zookeeper is to use this to achieve the distributed consistency. The Paxos algorithm, the raft protocol, and the Zab (Zookeeper Atomic Broadcast) protocol are all used to ensure the consistency of primary and standby data through a majority of votes.

ISR (In-sync replicas) mechanism: Kafka uses this mechanism to ensure data consistency. The ISR considers that for 2f+1 replicas, most polling mechanisms require a maximum of only f replicas to fail, and if you want to support fault tolerance for 2 replicas, you need to maintain a minimum of 5 replicas.

Communication......

About the author

Jing Er, graduated from the computer department of Northeastern University at the age of 20. After graduating from the first company due to outstanding language talent, in 1 years from the zero start to learn Japanese and ultra-high scores passed the international Japanese first-level examination, as a two-year Japanese translation work. After employment in Renren, transformation to do Internet development. Graduate student in psychology, CAs. There are nearly hundred technical invention patents, venture partner. We have technical support experience in Tokyo, Japan and Silicon Valley, USA. Currently the United States Group reviews Technical experts (Welcome to pay attention to the personal technology of static children: Programming Life), heart article can refer to my "New automation management of the training"

The technical exchange can pay attention to my github:github.com/xiexiaojing

Pay attention to the static children of the public number, irregular cartoon technology push ~