Java interface current limit algorithm

0. Preface

Common current-limiting algorithms are: Token bucket, leaky bucket. The counter can also be used for rough current limit implementations.

1. Algorithm Introduction 1.1 token bucket algorithm

The token bucket algorithm is a bucket that holds a fixed capacity token and adds a token to the bucket at a fixed rate. The token bucket algorithm is described as follows:

• Assuming a limit of 2r/s, add a token to the bucket at a fixed rate of 500 milliseconds;
• The bucket contains a maximum of B tokens, and when the bucket is full, the newly added token is discarded or rejected;
• When an n-byte packet arrives, n tokens are removed from the bucket, and the packet is sent to the network;
• If there are less than n tokens in the bucket, the token is not deleted and the packet is either dropped or the buffer waits.

1.2 Leaky bucket algorithm

As a metering tool (the leaky bucket algorithm as a Meter), the leaky bucket can be used for flow shaping (traffic Shaping) and flow control (trafficpolicing), and the leaky bucket algorithm is described as follows:

• A fixed-capacity leaky bucket that flows out of water droplets at constant fixed rates;
• If the bucket is empty, it does not need to flow out water droplets;
• Can flow into water droplets at any rate to the leaky bucket;
• If the inflow of water drops exceeds the capacity of the bucket, the incoming water droplets overflow (discarded) and the leaky bucket capacity is constant.

1.3 Summary

Comparison between token bucket and leaky bucket:

• The token bucket is the token that is added to the bucket at a fixed rate, whether the request is handled needs to see if the token is sufficient in the bucket, and the new request is rejected when the number of tokens is reduced to zero;
• The leaky bucket is a constant fixed rate of outgoing requests, the incoming request rate is arbitrary, when the number of incoming requests accumulated into the leaky bucket capacity, the new incoming request is rejected;
• The token bucket limit is the average rate of inflow (allow burst requests, as long as there is a token can be processed, support to take 3 tokens, 4 tokens), and allow a certain degree of burst traffic;
• The leaky bucket limit is the constant outflow rate (that is, the outflow rate is a fixed constant value, such as all the rate of 1 outflow, not one time is 1, the next is 2), thus smoothing burst inflow rate;
• The token bucket allows a certain degree of burst, while the main purpose of the leaky bucket is to smooth the inflow rate;
• Two algorithm implementations can be the same, but the direction is the opposite, for the same parameters obtained by the current limit effect is the same.

In addition, sometimes we use counters to limit the current, mainly to restrict the total number of concurrent, such as database connection pool, thread pool, the number of seconds to kill the concurrency, as long as the total number of global requests or the total number of requests for a certain period of time to set the threshold limit, is a simple rough total amount of current limit, rather than the average

2. Algorithm implementation

The current-limiting approach in the Java interface limit summary does not respond well to burst requests, where instantaneous requests may be allowed to cause problems, so in some scenarios it is necessary to reshape the burst request, which is processed as an average rate request processing (such as 5r/s, which processes a request every 200 milliseconds, Smoothed the rate). At this time there are two algorithms that satisfy our scenario: the token bucket and the leaky bucket algorithm. The guava framework provides a token bucket algorithm implementation that can be used directly.

Guava Ratelimiter provides a token bucket algorithm implementation: Smooth burst current limit (smoothbursty) and smooth warm-up current limit (Smoothwarmingup) implementation.

2.1 Smoothbursty

RateLimiter limiter = RateLimiter.create(5);System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());

You will get output similar to the following:

0.00.1989030.1954630.197120.1992520.196105

1, Ratelimiter.create (5) indicates a bucket capacity of 5 and added 5 tokens per second, that is, every 200 milliseconds to add a token;
2, Limiter.acquire () means to consume a token, if there is enough token in the current bucket to succeed (return value is 0), if there is no token in the bucket pause for a period of time, such as the order interval is 200 milliseconds, Wait 200 milliseconds before going to the consumption token (as the test case returned 0.198239, almost waiting for a 200-millisecond bucket for tokens to be available), an implementation that averages the burst request rate to a fixed request rate.

Look again at a burst example:

RateLimiter limiter = RateLimiter.create(5);System.out.println(limiter.acquire(5));System.out.println(limiter.acquire(1));System.out.println(limiter.acquire(1));

You will get output similar to the following:

0.00.9987290.19379

Limiter.acquire (5) indicates that the bucket has a capacity of 5 and a new 5 tokens per second, the token bucket algorithm allows a certain amount of bursts, so you can consume 5 tokens at a time, but the next Limiter.acquire (1) will wait for almost 1 seconds to have a token in the bucket, And the subsequent request is also shaped to a fixed rate.

RateLimiter limiter = RateLimiter.create(5);System.out.println(limiter.acquire(10));System.out.println(limiter.acquire(1));System.out.println(limiter.acquire(1));

You will get output similar to the following:

0.01.9989220.19615

Similar to the above example, the first second burst 10 requests, the token bucket algorithm also allows this burst (allow consumption of future tokens), but the next Limiter.acquire (1) will wait for almost 2 seconds in the bucket to have tokens, and the subsequent request is also shaped to a fixed rate.

Let's look at a sudden example:

RateLimiter limiter = RateLimiter.create(2);System.out.println(limiter.acquire());Thread.sleep(2000L);System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());System.out.println(limiter.acquire());

Let's look at a sudden example:

0.00.00.00.00.4997380.496078

1, the creation of a bucket capacity of 2 and 2 new tokens per second;
2, first call Limiter.acquire () to consume a token, at this time the token bucket can be satisfied (the return value is 0);
3, then the thread pauses for 2 seconds, the next two limiter.acquire () can be consumed to the token, the third limiter.acquire () also consumed to the token, to the fourth of the need to wait 500 milliseconds.

Here we can see that the bucket capacity we set is 2 (that is, the allowable burst), because there is one parameter in Smoothbursty: Maximum burst seconds (maxburstseconds) default is 1s, burst/bucket capacity = rate *maxburstseconds, So this sample bucket capacity/burst is 2, the first two in the example is the amount of burst that was accumulated before consumption, and the third start is the normal calculation. The token bucket algorithm allows tokens that are not consumed for a period of time to be staged into the token bucket for future use and allow future requests for such bursts.

Smoothbursty calculates the time of the next new token with the average rate and the time of the last new token, plus a bucket that is not used for a period of time (that is, the number of tokens that can burst). In addition, Ratelimiter provides a Tryacquire method for token consumption that is not blocked or can be timed out.

Because smoothbursty allow a certain degree of sudden, there will be some fear that if this burst is allowed, assuming that a large amount of traffic suddenly came, then the system is likely to carry this sudden. Therefore, a smoothing rate limiting tool is required, so that the system slowly tends to the average fixed rate after cold start (that is, the initial rate is smaller, then slowly tending to the fixed rate we set). Guava also provides smoothwarmingup to implement this requirement, which can be thought of as a leaky bucket algorithm, but it is not quite the same in some special scenarios.

2.2 Smoothwarmingup

Smoothwarmingup Creation Method: Ratelimiter.create (Doublepermitspersecond, long warmupperiod, timeunit unit)
Permitspersecond represents the number of tokens added per second, and Warmupperiod represents the interval between the transition from a cold boot rate to the average rate.
Examples are as follows:

RateLimiter limiter = RateLimiter.create(5, 1000, TimeUnit.MILLISECONDS);for(int i = 1; i < 5;i++) {    System.out.println(limiter.acquire());}Thread.sleep(1000L);for(int i = 1; i < 5;i++) {    System.out.println(limiter.acquire());}

You will get output similar to the following:

0.00.5189160.3567430.2191830.00.5199270.3547510.214637

The rate is the trapezoid rise rate, that is, cold start at a relatively large rate slowly to the average rate, and then tend to the average rate (trapezoid down to the average rate). The Warmupperiod parameter can be adjusted to start with a smooth fixed rate.

Some of the approaches to this application-level current limit are covered. Assuming that the application is deployed to multiple machines, application-level throttling is only a request-limiting flow within a single application and cannot be fully restricted. Therefore, we need distributed current limit and access layer limit flow to solve this problem.

Reference:
http://jinnianshilongnian.iteye.com/blog/2305117

Java interface current limit algorithm