Interface throttling practices

I. Problem description

One day, Mr. A suddenly found that his Interface request volume suddenly rose to 10 times of the previous one. Not long ago, the interface was almost unavailable and triggered A chain reaction, causing the entire system to crash. How can this problem be solved? Life has given us the answer: for example, old-fashioned electric locks are installed with fuses. Once someone uses ultra-high-power equipment, fuses will be burned out to protect all electrical appliances from burning out by strong current. Similarly, our interfaces also need to be installed with fuses to prevent unexpected requests from causing system paralysis due to excessive system pressure. When the traffic is too high, A denial or redirection mechanism can be adopted.

II. Common traffic limiting algorithms

There are two common throttling algorithms: the bucket leakage algorithm and the token bucket algorithm. This article gives a clear introduction (analysis of the overload protection algorithm ).

The idea of the bucket leakage algorithm is very simple. The request is first sent to the bucket that leaks water at a certain speed. When the water request passes through the conference, it directly overflows, it can be seen that the bucket leakage algorithm can effectively limit the data transmission rate.

Figure 1 bucket leakage algorithm

In many application scenarios, in addition to limiting the average data transmission rate, it is also required to allow burst transmission to some extent. At this time, the bucket leakage algorithm may not be suitable, and the token bucket algorithm is more suitable. 2. The principle of the token bucket algorithm is that the system will put the token into the bucket at a constant speed. If the request needs to be processed, you must first obtain a token from the bucket, if no token is available in the bucket, the service is denied.

Figure 2 token bucket algorithm

III. Throttling tool RateLimiter

Google open-source toolkit guava provides the throttling tool class RateLimiter, which is based on the "token bucket algorithm" and is very convenient to use. For more information about the interfaces of this class, see RateLimiter. For more information about how to use RateLimiter, see RateLimiter usage.

The main source code is as follows:

Public double acquire () {return acquire (1);} public double acquire (int permits) {checkPermits (permits); // check whether the parameter is valid (greater than 0) long microsToWait; synchronized (mutex) {// synchronous microsToWait = reserveNextTicket (permits, readSafeMicros () to cope with concurrency; // get the time to wait} ticker. sleepMicrosUninterruptibly (microsToWait); // wait. When the limit is not reached, microsToWait is 0 return 1.0 * microsToWait/TimeUnit. SECONDS. toMicros (1L);} private long reserveNextTicket (double requiredPermits, long nowMicros) {resync (nowMicros); // add the token long token = signature-nowMicros; double storedPermitsToSpend = Math. min (requiredPermits, this. storedPermits); // Obtain the number of tokens consumed by this request. double freshPermits = requiredPermits-storedPermitsToSpend; long waitMicros = storedPermitsToWaitTime (this. storedPermits, storedPermitsToSpend) + (long) (freshPermits * stableIntervalMicros); this. nextFreeTicketMicros = nextFreeTicketMicros + waitMicros; this. storedPermits-= storedPermitsToSpend; // subtract the consumed token return microsToNextFreeTicket;} private void resync (long nowMicros) {// if nextFreeTicket is in the past, resync to now if (nowMicros> nextFreeTicketMicros) {storedPermits = Math. min (maxPermits, storedPermits + (nowMicros-nextFreeTicketMicros)/stableIntervalMicros); nextFreeTicketMicros = nowMicros ;}}

 

 

 

Interface throttling practices