A little thought on the control of high concurrency traffic

a little thought on the control of high concurrency traffic
"Foreword" in the actual project, had encountered the peak of the line 5w+qps, also in the pressure measurement state experienced 10W+QPS Large flow request, the topic of this blog is his own high concurrency traffic control of a little thought.
some ideas for coping with large flow

First of all, let's take a look at what is big traffic.

Large flow, we are likely to emerge: TPS (transaction volume per second), QPS (requests per second), 1w+,5w+,10w+,100w+ .... In fact, there is no absolute number, if the amount of pressure caused by the system, affecting the performance of the system, then this amount can be called the large flow.

Second, what are some common means of coping with large flows.

Cache: Plainly, let the data into the cache as early as possible, close to the program, do not have a large number of frequent access to the DB.

Downgrade: If it's not the core link, downgrade the service. To make a metaphor, now the app is about thousand people face, get the data, do a personalized display, if in the large flow, this sort can be degraded.

Current limit: As we all know, Beijing Subway early peak, subway station will do one thing, is limited flow. The idea is straightforward, is to limit the request within a certain period of time, to ensure that the system is not washed out, while maximizing the system's throughput.

Notice that, in some cases, the cache and downgrade is not solve the problem, for example, the e-commerce 11, the user's purchase, the next single behavior, is involved in a large number of write operations, and is the core link, can not be degraded, this time, limit the current is more important.

So next, let's focus on the current limit.
common ways to limit current

Current limiting methods are: counters, sliding windows, leaky barrels, tokens.

Counter

Counter is a relatively simple current-limiting algorithm, the use of a wide range, at the interface level, many places use this way to limit the flow. Over a period of time, the counter is counted, compared with the threshold, to the time critical point, the counters clear 0.

It is important to note that there is a time-critical problem. For a chestnut, there was no user request during the period from 12:01:00 to 12:01:58, then 100 requests were made at 12:01:59, OK, then 100 requests were made at the moment of 12:02:00. Here you should be able to feel that at this point it can withstand a large number of requests from malicious users, even exceeding the expected endurance of the system.

Sliding window

Since the counter has a critical point flaw, a sliding window algorithm has emerged to solve it.

The sliding window means that the fixed time slices are divided and moved with the passage of time, which subtly avoids the counter critical point. In other words, the fixed number of movable grids will be counted to determine the threshold, so the number of squares affects the accuracy of the sliding window algorithm.

Leaky bucket

Although the sliding window effectively avoids the problem of time critical point, but still has the concept of time slices, and the leaky bucket algorithm in this respect than sliding window, more advanced.

There is a fixed bucket, the rate of water intake is uncertain, but the rate of water is constant, when it is full, it will overflow.

Token bucket

Note that the water flow rate of the leaky bucket is constant, which means that if there is an instantaneous large volume, most of the requests will be discarded (that is, the so-called overflow). To solve this problem, the token bucket has been improved by the algorithm.

The speed at which tokens are generated is constant, and there is no speed limit to the request to fetch the token. This means that in the face of instantaneous large traffic, the algorithm can request a large number of tokens in a short period of time, and the process of taking tokens is not a very expensive thing. (A little production token, the meaning of the consumer token)

Whether for the token bucket can not get the token is rejected, or leaking buckets of water overflow, are to ensure that most of the normal use of traffic, and the loss of a small portion of traffic, this is reasonable, if because a very small amount of traffic needs to be guaranteed, then it may cause the system to reach the limit and hang off, outweigh the gains.

Current limit artifact: Guava Ratelimiter

Guava is not only powerful in terms of collections, caches, asynchronous callbacks and so on, but also encapsulates the API for limiting the flow.

Guava Ratelimiter based on the token bucket algorithm, we just need to tell ratelimiter the system limits the amount of QPS, then Ratelimiter will be at this speed into the bucket inside the token, and then request, by Tryacquire () method to obtain a license (token) from the Ratelimiter.

Current limiting in distributed scenarios

Some of the methods mentioned above are for single-machine, in fact, most of the scenarios, single-machine current limit is enough. The means of distributed low-limit flow often require a combination of many technologies, such as Nginx+lua,redis+lua. This article mainly discusses the single-machine current limit, which is not in detail in the distribution scenario under the current limit.

In a word, let the system flow, first to queue, limit flow, do not let traffic directly to the system.

Source: http://blog.51cto.com/zhangfengzhe/2066683

Copyright statement: The content source network, the copyright belongs to the original person. Unless it is impossible to confirm, we will mark the author and source, and if there is any infringement please inform us that we shall immediately delete and apologize. Thank you.