About two current-limiting modes

In the traffic warning and current limiting schemes, there are two common methods. The first sliding window mode is controlled by counting the number of accesses over a period of time, and a current limit is reached when the number of accesses reaches a certain peak. The second is the number of concurrent users mode, by controlling the maximum number of concurrent users, to achieve the purpose of traffic control. The following two simple analysis of the pros and cons.

1. sliding window Mode

Pattern Analysis:

Each time a visit comes in, we determine whether the total amount of traffic in the first n units of time exceeds the set threshold, and the number of requests on the current time slice is +1.

Each format represents a fixed time (such as 1s), a counter for each grid, we want to obtain the first 5s of the request volume, is the current time slice I ~ i-4 time on-chip counter to accumulate.

This mode is implemented in a way that is more closely aligned with the essential meaning of flow control. Understanding is simple. However, due to the unpredictability of traffic, a large number of requests for the first half of the unit time are poured in, and the second half rejects all requests. (Usually, you need to be able to reduce the unit time to a small enough to alleviate) Secondly, it is difficult to determine how much of this threshold setting is appropriate, only through experience or simulation (such as pressure measurement) to estimate, even the pressure measurement is difficult to estimate the accuracy. The hardware parameters of each machine in the cluster deployment are different, which may cause us to set the threshold value for each machine differently. The same machine at different time points of the system pressure is not the same (such as the evening there are some tasks, or the impact of some other business operations), can withstand the maximum threshold value is not the same, we can not consider the comprehensive.

Therefore, the sliding window mode is usually applicable to the need for protection of a resource (or the promise is more appropriate: I have committed to the provider of an interface, the maximum amount is not more than XX), such as the protection of the DB, the control of a call to a service.

Code Implementation ideas:

Each time slice (unit time) is a separate counter that is used to save the array. Maps the current time in some way (for example, modulo) to an item in an array. Each access to the current time slice on the counter +1, and then calculate the sum of the first n time slices of traffic, exceeding the threshold is limited to flow.

Java code

2. /**
4. * The realization of sliding window
6. * @author Shimig
8. *
10. */
12. Public class Slidingwindow {
16. / * Loop queue * /
18. private volatile atomicinteger[] timeslices;
20. / * Total length of queue * /
22. Private volatile int timeslicesize;
24. / * Duration of each time slice * /
26. Private volatile int timemillisperslice;
28. / * Window length * /
30. Private volatile int windowsize;
34. / * Current time slice position * /
36. private Atomicinteger cursor = new Atomicinteger (0);
40. Public Slidingwindow (int timemillisperslice, int windowsize) {
42. this.timemillisperslice = Timemillisperslice;
44. this.windowsize = windowsize;
46. //guaranteed to be stored at least two windows
48. this.timeslicesize = windowsize * 2 + 1;
50. }
54. /** 
56. * Initialize the queue, because this initialization will request some content space, in order to save space, deferred initialization
58. */
60. private void Inittimeslices () {
62. if (timeslices! = null) {
64. return;
66. }
68. //In multi-threaded case, there will be multiple initialization situation, it's okay
70. ///We only need to ensure that the obtained value must be a stable, all of which are initialized first, the last assignment method
72. atomicinteger[] localtimeslices = new Atomicinteger[timeslicesize];
74. For (int i = 0; i < timeslicesize; i++) {
76. Localtimeslices[i] = new Atomicinteger (0);
78. }
80. Timeslices = localtimeslices;
82. }
86. private int Locationindex () {
88. Long time = System.currenttimemillis ();
90. return (int) ((time/timemillisperslice)% timeslicesize);
92. }
96. /** 
98. * <p> Count +1 on the time slice and return the sum of all the counts in the window
100. * <p> This method will always be +1 for a time slice if it is called
102. *
104. * @return
106. */
108. public int incrementandsum () {
110. Inittimeslices ();
112. int index = Locationindex ();
114. int sum = 0;
116. //cursor equals index, returns True
118. //cursor is not equal to index, returns FALSE, and the cursor is set to index
120. int oldcursor = cursor.getandset (index);
122. if (oldcursor = = index) {
124. //Continue in the current time slice +1
126. Sum + = Timeslices[index].incrementandget ();
128. } Else {
130. //There may be other thread has been placed 1, the problem is not very
132. Timeslices[index].set (1);
134. //clear, there will be time slice jumping when the traffic is not big
136. Clearbetween (oldcursor, index);
138. //Sum + = 0;
140. }
142. For (int i = 1; i < windowsize; i++) {
144. Sum + = timeslices[(index-i + timeslicesize)% timeslicesize].get ();
146. }
148. return sum;
150. }
154. /** 
156. * To determine whether access is allowed, not exceeding the threshold, will be a time slice +1
158. *
160. * @param threshold
162. * @return
164. */
166. public boolean allow (int threshold) {
168. Inittimeslices ();
170. int index = Locationindex ();
172. int sum = 0;
174. //cursor is not equal to index, cursor is set to index
176. int oldcursor = cursor.getandset (index);
178. if (oldcursor! = index) {
180. //There may be other thread has been placed 1, the problem is not very
182. Timeslices[index].set (0);
184. //clear, there will be time slice jumping when the traffic is not big
186. Clearbetween (oldcursor, index);
188. }
190. For (int i = 1; i < windowsize; i++) {
192. Sum + = timeslices[(index-i + timeslicesize)% timeslicesize].get ();
194. }
198. //Threshold value judgment
200. if (sum <= threshold) {
202. //does not exceed threshold value only +1
204. Sum + = Timeslices[index].incrementandget ();
206. return true;
208. }
210. return false;
212. }
216. /** 
218. * <p> zeroing the time slice count between fromindex~toindex
220. * <p> in extreme cases, when the loop queue has gone more than 1 timeslicesize above, the Qing 0 here cannot be carried out as expected
222. *
224. * @param fromIndex not included
226. * @param toindex not included
228. */
230. private void Clearbetween (int fromIndex, int toindex) {
232. For (int index = (fromIndex + 1)% Timeslicesize; Index! = toindex; index = (index + 1)% timeslicesize) {
234. Timeslices[index].set (0);
236. }
238. }
242. }

2, concurrent user number mode

Pattern Analysis:

Each time the operation executes, we determine whether to limit the flow by judging whether the number of accesses currently executing exceeds a certain threshold.

This mode looks at the alternative of thinking comparison, but it has its unique. In fact, the root of our current limit is to protect the resources, to prevent the system to accept too many requests, overwhelmed, slowing down the service of other interfaces in the system, causing avalanches. What we really need to be concerned about is the requests that are running, and those that have already been completed are past and no longer need to be cared for.

Let's take a look at how the threshold is calculated, for a request, the response time RT, QPS is a relatively easy to obtain parameters, then we calculate: Qps/1000*rt.

In addition, an application is often a complex system that provides more than one service or exposed requests and resources. Internal GC, execution of timed tasks, surges in other service accesses, external relying parties, DB jitter, or inadvertently a bug in the code. Can lead to changes in response time, resulting in changes in system performance capacity. This mode, the appropriate automatic adjustment, when the system is not, RT increases, will automatically adapt to the QPS.

Code Implementation ideas:

When the access starts, we are on the current counter (atomic counter) +1, when finished,-1. The counter is the number of requests that are currently executing. Just determine if the counter exceeds the threshold value.

About two current-limiting modes