High concurrency Architecture Detailed _ Architecture

First, what is high concurrency

High concurrency is one of the factors that must be considered in the architecture design of the Internet Distributed system, it usually means that the system can handle many requests simultaneously in parallel through the design.

Some commonly used indicators of high concurrency are response times (Response time), throughput (throughput), query rate per second QPS (query/Second), number of concurrent users, etc.

Response time: The time the system responds to requests. For example, the system needs 200ms to process an HTTP request, and this 200ms is the response time of the system.

Throughput: The number of requests processed during the unit time.

QPS: Number of response requests per second. In the Internet arena, this metric is not as obvious as the throughput.

Concurrent users: The number of users who also host the normal use of system features. For example, an instant messaging system, while online to a certain extent, represents the number of concurrent users of the system.

Second, how to enhance the system's concurrency capability

Internet distributed architecture design, improve the system concurrency capacity of the way, there are two main methodologies: vertical expansion (Scale up) and horizontal expansion (Scale out).

Vertical expansion: Enhance the capacity of stand-alone processing. There are two ways to expand vertically:

(1) Enhance the performance of stand-alone hardware, such as: increase the number of CPU cores such as 32 cores, upgrade a better network card such as Million trillion, upgrade a better hard drive such as SSD, expansion of the hard disk capacity such as 2T, expansion of the system memory such as 128G;

(2) Improve the performance of stand-alone architecture, such as: Use cache to reduce IO times, use asynchronous to increase the single service throughput, use the lock-free data structure to reduce response time;

In the early days of the rapid development of the Internet business, if the budget is not a problem, it is strongly recommended to use the "enhanced stand-alone hardware performance" approach to improve system concurrency, because the company's strategy is often to develop business clock, and "enhance stand-alone hardware performance" is often the fastest method.

Whether it is to improve the performance of stand-alone hardware, or to upgrade the performance of stand-alone architecture, there is a fatal problem: there is always a limit to the performance of stand-alone. So the Internet distributed architecture Design high concurrent ultimate solution or horizontal expansion.

Horizontal expansion: As long as the number of servers increased, you can linearly expand system performance. Horizontal scaling is a requirement for system architecture design, how to design horizontally in various layers of architecture, and the common level extension practice of Internet company architecture are the main contents of this paper.

Third. Common Internet layered architecture

Common Internet distributed architectures, as above, are divided into:

(1) Client layer: Typical caller is browser browser or mobile app

(2) Reverse proxy layer: System entry, reverse proxy

(3) Site Application layer: Implement core application logic, return HTML or JSON

(4) Service layer: If the implementation of the service, there is this layer

(5) data-cache layer: Cache acceleration Access Storage

(6) Data-database layer: Database solidification data storage

The level of the whole system at all levels of expansion, and how the implementation of the different.

Fourth, layered horizontal extended architecture practice

Horizontal extension of the reverse proxy layer

The horizontal extension of the reverse proxy layer is achieved through "DNS polling": Dns-server for a domain name configured with multiple resolution IP, each DNS resolution request to access Dns-server, will poll to return these IP.

When Nginx become a bottleneck, as long as the number of servers, add new Nginx service deployment, add an extranet IP, you can expand the performance of the reverse proxy layer, to achieve the theoretical infinite high concurrency.

Horizontal extension of the site layer

The horizontal extension of the site layer is achieved through "Nginx". By modifying nginx.conf, you can set up multiple Web backend.

When the Web backend becomes the bottleneck, as long as the number of servers, new Web services deployment, in the Nginx configuration of the new Web backend, you can extend the performance of the site layer, so that theoretically unlimited high concurrency.

Horizontal extension of the service layer

The horizontal extension of the service layer is achieved through the service connection pool.

When the site layer calls downstream service layer Rpc-server through rpc-client, the connection pool in Rpc-client establishes multiple connections to downstream services, and when the service becomes the bottleneck, as long as the number of servers is increased, new services are deployed, By establishing a new downstream service connection at Rpc-client, we can extend the performance of the service layer and achieve the theoretically infinite concurrency. If you need to gracefully carry out the service layer automatic expansion, here may need to configure the Heart Service automatic discovery function support.

Horizontal extension of the data tier

In the case of large amount of data, the data layer (cache, database) involves the horizontal extension of data, and the data (cache, database) that is stored on one server is split horizontally to different servers to achieve the purpose of extending the system performance.

There are several common horizontal split methods in the Internet data tier, taking the database as an example:

Split horizontally by range

For each data service, a range of data is stored, as in the example above:

USER0 Library, storage uid range 1-1kw

User1 Library, storage uid range 1kw-2kw

The benefits of this program are:

(1) Simple rules, service only judge the UID range can be routed to the corresponding storage services;

(2) Good data equalization;

(3) relatively easy to expand, can be added at any time a uid[2kw,3kw] data services;

The disadvantages are:

(1) The requested load is not necessarily balanced, in general, the new registered users will be more active than the old users, large range of service request pressure will be greater;

Split horizontally by Hash

For each database, store some of the data in the hash of a key value.

USER0 Library, storing even-UID data

User1 library, storing odd UID data

The benefits of this program are:

(1) The rule is simple, the service only can be routed to the corresponding storage service with the UID hash;

(2) Good data equalization;

(3) The uniformity of the request is better;

The disadvantages are:

(1) It is not easy to expand, expand a data service, the hash method changes, may need to carry out data migration;

It should be noted that the way to augment the performance of the system by horizontal split is fundamentally different from the master-slave synchronous read-write separation.

Extend database performance horizontally by splitting:

(1) The amount of data stored on each server is the total amount of 1/n, so the performance of single machine will also be upgraded;

(2) N server data does not intersect, the server on the collection of data is the complete set of data;

(3) The level of data split to n servers, theoretically read performance expansion of n times, write performance has been expanded by N times (in fact, far more than n times, because the number of stand-alone data for the original 1/n);

Extended database performance through master-slave synchronous read-write separation:

(1) The amount of data stored on each server is the same as the total amount;

(2) N server data are the same, are complete works;

(3) Theoretically read performance expanded n times, writing is still a single point, write performance unchanged;

The horizontal split of the cache layer is similar to the horizontal split of the database layer, and is mostly in the form of range splitting and hashing, and is no longer expanded.

Fifth. Summary

High concurrency is one of the factors that must be considered in the architecture design of the Internet Distributed system, it usually means that the system can handle many requests simultaneously in parallel through the design.

There are two main ways to improve the system concurrency capability: Vertical expansion (Scale up) and horizontal expansion (Scale out). The former vertical expansion can increase the performance of stand-alone hardware, or improve the performance of stand-alone architecture to improve concurrency, but stand-alone performance is always limited, the Internet distributed architecture Design high concurrent ultimate solution or the latter: horizontal expansion.

In the Internet layered architecture, the practice of horizontally expanding at different levels is different:

(1) The reverse proxy layer can be extended horizontally through the way of "DNS polling";

(2) The site layer can be expanded horizontally through nginx;

(3) The service layer can be expanded horizontally through the service connection pool;

(4) The database can be expanded horizontally according to the data range, or the way of data hashing;

After the implementation of horizontal expansion of each layer, can increase the number of servers in the way to improve the performance of the system, so that the theoretical performance is unlimited.

Original address: Architect's path (no connection, put on his two-dimensional code)