The internal strength of distributed system architecture

The road of the day, the loss of surplus and insufficient, is the false victory, not enough to win.

For the software architecture, more is a kind of thought, namely the internal strength to fix. At the level of Tao and operation, it is more emphasis on the cultivation of Tao, the depth of Tao determines the realm of architecture. Relatively speaking, the technique is a means, with different circumstances came into being, like the Taiji sword law and solitary nine swords, can do with the environment and change.

Architecture is a trade-off

No one architecture can be applied to all environments, nor is there a technology or framework that solves all problems, and there are often multiple solutions for the same scenario. At the time of architecture, there is a contradiction between the tradeoffs of schemes and tools, such as high availability, high concurrency, consistency itself, and whether asynchronous is synchronous, requires transactions, how transactions are applied, caching, splitting, disaster recovery, publishing, and so on, each of which needs to be weighed from a variety of technical implementations. Refinement to the framework, ActiveMQ, ROCKETMQ, Kafka, Redis, zookeeper and so on can implement the Message Queuing model, the specific use of which will need to combine the scene to weigh.

Division and Hopewell

The world big event, the time must be divided, the time will be combined, in solving the problem of high concurrency distributed most of the use of the idea of separation and co-operation.

When the amount of data is large, concurrency is a lot of time to consider splitting (divide and conquer), such as layered design, horizontal splitting, vertical splitting, sub-IDC, sub-database sub-table ... Wait a minute. And these splits themselves are layered, for example, the DNS layer can assign traffic by geography or operator to different IDC, the business layer can assign the business processing logic to multiple subsystems, the system layer can be split horizontally according to the user, and the storage layer can assign data to different libraries according to the rules different tables In addition, read-write separation, hotspot separation, and independent out-of-cache layer also embody the idea of distributed system architecture.

Why do so many split, split is to make more for less, in the case of a limited single-node processing capacity, through the horizontal split to provide wireless expansion capacity, when the huge traffic through the split, each node to deal with the QPS will be reduced; split is to simplify the complexity of single node, The current microservices (of course, the service governance that the microservices cause needs to be said), and the two-phase transaction submission, is the means of decomposing the complexity of a single node through the multi-dimensional splitting of the complex business.

If you split it up, Hadoop will decompose complex tasks into a mapreduce job processing and aggregation, and the processing efficiency is greatly improved, and this decomposition must be accompanied by aggregation. In some business scenarios, the two types of nodes are frequently called each other, and by merging the original RPC calls into local JVM calls, the system performance can be greatly improved.

Isolation

Isolation is also a kind of thought, which also contains the meaning of the points, such as gray scale, pressure test isolation, static and dynamic isolation, multi-release and so on.

Transformation

Routing and transformation, can be seen as a derivative of the idea. Too many points, you need to be able to forward the request to the correct location, in addition to the various communication formats and protocols to convert.

Repetition and Uniqueness (idempotent)

Idempotent is accompanied by the request of a set of delivery, there may be a number of scenarios when sending a request: The receiving end is not necessarily receive, the receiver can only receive one time, the receiving end can be repeatedly received. For the configuration push platform, most scenarios require the receiving side to receive the configuration information repeatedly, but only the last message can be saved, and for a payment request, if sent repeatedly, the receiving side to control can only be processed once (that is, to power control).

Ultimately consistent

In the Distributed system architecture, there are two kinds of choices in dealing with transactional business, one is distributed transaction and one is final consistency processing. Distributed transaction control is more complex, generally using two-phase commit method, the idea is either success or failure, attention or failure is that there is a step error in all rollback, seemingly to ensure that the data consistency but the actual business has failed. The core idea of final consistency relative to distributed transactions is to make the business as successful as possible, that is, when there may be intermediate step failures in the business process, but by compensating logic to ensure that the failed steps continue to execute, it is time to mark the business success as much as possible.

Parallel and serial

Spreading large amounts of computing across multiple nodes, multiple processes, and multiple threads for parallel processing is a common technique for dealing with big data calculations. And the parallel is the serial (or the concurrent lock) processing, in the distributed environment, the concurrent lock can be achieved through the database, zookeeper and so on.

Active and passive

Active vs. passive is primarily a scenario for how clients get server-side content, typically push-mode (push) and pull-mode in Message Queuing. The implementation of the specific underlying, no outside polling, long links, and so on, in which the use of polling is not necessarily a pull model, many of the push model of its bottom is also through polling implementation.

Synchronous vs. asynchronous

Synchronous and asynchronous are often the things to be faced with, asynchronous can reduce the system congestion, such as AJAX, Message Queuing (can also achieve the role of Peak valley) and so on. In addition, streaming and off-line computing can also be seen as synchronous and asynchronous derivative technologies. Another deep level, will be derived from batch computing, the whole amount, increment and other ideas.

Donuts

Current limit: When a large amount of traffic is called, the whole system can be ensured by sacrificing some request processing.

Downgrade: When faced with traffic spikes (such as big promotion) tend to stop the secondary services, in order to save resources to the main services; In addition, when a service is unavailable, returning the default result directly is a performance of the downgrade;

Fuse: For the financial system, when a bug can cause a loss of capital, at this time need to bypass the system to constantly check, when found abnormal can be timely termination of processing.

For the above several aspects, the system should be able to do automatic current limit, automatic demotion and automatic fuse.

At last

All of the above problems are due to the amount (data volume, request volume, etc.) produced, the equivalent of a very small number of hours do not need the architecture, by increasing the memory, CPU, machine and so can be resolved; the equivalent to a certain extent, the need to consider the architecture, and the architecture of the Tao and the operation is "Tao Can road, very