Discussion on TCP congestion control algorithm

TCP by maintaining a congestion window for congestion control, congestion control principle is that as long as there is no congestion in the network, the value of the congestion window can be increased some more, in order to send more data packets, but as long as the network congestion, the congestion window should be reduced the value of some, to reduce the number of packets injected into the network.

In the process of TCP congestion control algorithm development, there are several different ideas:

• Congestion control based on packet loss: The packet loss is considered as congestion, and the congestion window is gradually increased by slow detection, and when packet loss occurs, the congestion window is reduced, such as Reno, Cubic and so on.
• Delay-based congestion control: The delay is increased as congestion, the delay increases the congestion window, the delay decreases when the congestion window, such as Vegas, Fasttcp and so on.
• Congestion control based on link capacity: real-time measurement of network bandwidth and latency, which is considered to be congestion when the total amount of network escalation is greater than the bandwidth delay product, such as BBR.
• Learning-based congestion control: there is no specific congestion signal, but the use of evaluation functions, based on training data, using machine learning method to form a control strategy, such as Remy.

The core of congestion control algorithm is to choose an effective strategy to control the change of congestion window, and some classical congestion control algorithms are introduced below.

Vegas

VEGAS[1] Increases the latency of the RTT as the network congestion signal, the RTT increases, the congestion window decreases, the RTT decreases, and the congestion window increases.  Specifically, Vegas adjusts the size of the congested window by comparing the actual throughput and expected throughput, the expected throughput: expected = Cwnd/basertt, actual throughput: Actual = Cwnd/rtt,diff = (expected-actual) * Basertt,basertt is the minimum value for all observed response times, typically the rtt,cwnd of the first packet sent after the connection is established is the size of the current congested window. Vegas defines two thresholds, A/b, the Congestion window decreases when diff > B., and when a <= diff <=b, the Congestion window is not changed, and when diff < A, the congestion window increases.

The Vegas algorithm uses the change of RTT to judge the available bandwidth of the network, and can accurately measure the available bandwidth of the network, and the efficiency is better. However, in the case of coexistence of Vegas and other algorithms in the network, the congestion control algorithm based on packet loss will attempt to fill the buffer in the network, resulting in an increase in the rtt of the Vegas computation, thus reducing the congestion window, which makes the transmission speed more and more slow, so Vegas is not universally used on the Internet.

Application scenario: Applicable to the network only exists Vegas a congestion control algorithm, the situation of competition fairness.

Reno

RENO[2] The congestion control process is divided into four stages: slow start, congestion avoidance, fast retransmission and fast recovery, are the basis of many existing congestion control algorithms, the following detailed description of these stages.

Slow start phase, in the absence of packet loss each received an ACK will be the congestion window size plus one (unit is MSS, the maximum single message length), each round of the sending window increases by one times, exponential growth, if the packet drops, the congestion window is halved, into the congestion avoidance phase; When the window reaches the slow boot threshold or drops, Into the congestion avoidance phase, the window is added one at a time, linear growth, when the receipt of a message of three duplicate ACK, the message is considered to be the next message is lost, into the fast retransmission phase, immediately re-transmit the lost message, rather than wait for the retransmission of time-out; Modify the slow boot threshold to half the current congestion window value, and the congestion window value equals the slow start threshold, then enter the congestion avoidance phase and repeat the appeal process. The Reno congestion control process is shown in 1.

Figure 1, TCP Reno congestion control process

The Reno algorithm will receive an ACK signal as the basis of the congestion window growth, in the early low-bandwidth, low latency network can play a good role, but with the increase in network bandwidth and delay, the shortcomings of Reno gradually manifested, the sender from the delivery of the paper to receive an ACK through a RTT, In High bandwidth latency (Hi Bandwidth-delay product,bdp) networks, the RTT is large, causing the congestion window to grow very slowly, and the transfer speed takes a long time to reach the maximum bandwidth, resulting in a low bandwidth utilization.

Application scenario: Suitable for low-latency, low-bandwidth networks.

Cubic

CUBIC[3] is the default TCP congestion control algorithm after Linux kernel 2.6, using a cubic function (Cubic functions) as the growth function of the congestion window, where c is the adjustment factor, T is the time from the last narrowing of the congestion window, Wmax is the window size of the last time the congestion occurred, and β is the multiplication reduction factor. As you can see from the function, the growth of the congestion window is no longer related to the RTT, but only depends on the maximum window and the interval value of the last congestion when the last congestion occurred.

Cubic Congestion window growth curve is as follows, convex curve part is stable growth stage, concave curve part is the maximum bandwidth detection stage. 2, at the beginning, the congestion window grew very fast, at the close to the Wmax port, the growth rate is smooth, avoid the traffic burst, resulting in packet loss; near Wmax, the congestion window no longer increases, and then slowly detects the network maximum throughput, guaranteeing stability (congestion is likely to occur near the Wmax). After being away from Wmax, increase the speed of the window to ensure the utilization of the bandwidth.

Figure 2, TCP Cubic congestion window growth function

When a packet loss occurs, the congestion window is multiplied to reduce the growth process. This approach allows the congestion window to remain near Wmax, thus ensuring bandwidth utilization. The cubic congestion control process is shown in 3:

Figure 3, TCP cubic congestion control process

The advantage of the cubic algorithm is that as long as there is no packet loss, it will not actively reduce its transmission speed, can maximize the use of network residual bandwidth, improve throughput, in high-bandwidth, low packet loss rate of the network can play a better performance.

However, cubic with the previous congestion control algorithm, can not distinguish between congestion packet loss and transmission error drops, as long as the discovery drops, will reduce congestion window, reduce the transmit rate, and in fact, transmission error drops when the network does not necessarily occur congestion, but the probability of transmission error drops is very low, Therefore, the performance impact of the cubic algorithm is not very large.

Another disadvantage of the cubic algorithm is that it is too aggressive to keep increasing the size of the congested window without dropping packets, injecting traffic to the network, filling the buffer of the network device, and bufferbloat (buffer bloat). Due to the long-term saturation state of the buffer, new packets entering the network will queue up in the buffer, increasing the unnecessary queue delay, the larger the buffer, the higher the delay. In addition, the cubic algorithm still increases the congestion window while increasing the bandwidth utilization, and increases the packet loss rate indirectly, causing the network jitter to intensify.

Application scenario: Suitable for high bandwidth, low packet loss rate network, can effectively use bandwidth.

BBR

BBR[4] is a new congestion control algorithm Google introduced in 2016, has been deployed on the YouTube server and Google across the data center WAN, according to official YouTube data, after the deployment of BBR, the delay in the global access to YouTube decreased by 53%, In developing countries with high latency, the delay was reduced by 80%. Currently, BBR has been integrated into the Linux version 4.9 kernel.

The BBR algorithm does not increase the packet loss or delay as a signal of congestion, but considers that congestion occurs when the total amount of packets on the network is greater than the product of the bottleneck link bandwidth and latency, so BBR is also called congestion-based congestion control algorithm (congestion-based congestion Control). The BBR algorithm periodically detects the capacity of the network, alternating the bandwidth maxima and latency minima over a period of time, and the product as the Congestion window size (alternating measurement is due to the fact that the maximum bandwidth and the pole-hour delay can not be obtained at the same time, the bandwidth is large when the network is filled up to cause queuing, the delay is necessarily great, When the delay is very small, the packet is not forwarded directly, the bandwidth is necessarily very small, so that the value of the congestion window always coincides with the capacity of the network.

Due to the BBR congestion window is precisely measured out, does not indefinitely increase the congestion window, will not fill the network device buffer, avoid the emergence of bufferbloat problems, so that the delay greatly reduced. As shown in 4, when the network buffer is filled, the 250ms,cubic algorithm will continue to increase the congestion window, so that the delay continues to increase to 500ms and the loss of packets, the entire process cubic has been in Gao Shiyan state, and BBR because will not fill the network buffer, the delay has been in a low state.

Figure 4, Cubic and bbr RTT comparison

Because the BBR algorithm does not drop packets as a congestion signal, so in the high packet loss rate of the network, BBR still has very high throughput, 5, in the network environment of 1% packet loss rate, cubic throughput has been reduced by more than 90%, and BBR throughput is almost unaffected, when the packet loss rate is greater than 15%, The throughput of the BBR is greatly reduced.

Figure 5, cubic and BBR transfer rates vs. packet loss rates

The BBR algorithm is a feedback-driven, self-regulating mechanism, independent of the TCP congestion control state machine control, by measuring the network capacity to adjust the congestion window, the transmission rate is self-control, while the traditional congestion control algorithm is only responsible for calculating congestion window, regardless of the transmission rate (pacing rates), How it is determined by TCP itself, which causes packets to queue or drop packets in the vicinity of the bottleneck bandwidth due to a spike in the sending rate.

After testing, in the high delay, high packet loss rate of the environment, BBR relative to the cubic algorithm in the transmission speed has a greater increase, the specific test results Table 1 shows:

Table 1 200ms cubic and BBR transmission speed comparison

The disadvantage of the BBR algorithm is that when the device queue cache is large, BBR may compete for more aggressive algorithms such as cubic, because BBR does not take the initiative to occupy the queue cache, if the cubic traffic takes up the queue cache for a long time, the BBR will increase the minimum RTT measured in multiple cycles. In turn, the bandwidth of the BBR is reduced.

Application scenario: Suitable for high bandwidth, Gao Shiyan, with a certain packet loss rate of long-fat network, can effectively reduce transmission delay, and ensure high throughput.

Remy

REMY[5] Also known as computer-generated congestion control algorithm (computer-generated Congestion-control algorithm), a model of congestion control algorithm is generated by machine learning method. By inputting various parameter models (such as bottleneck link rate, delay, number of senders on bottleneck chain, etc.), using a target function to quantitatively determine the merits and demerits of the algorithm, in the process of generating the algorithm, the Congestion window is adjusted in different ways for different network states, and the adjustment mode is modified repeatedly until the objective function is optimal. Finally, a mapping table is generated for the network state to the regulation mode, and the congestion window can be directly selected from the mapping table according to the specific network environment in the real network.

Remy tries to block the difference of the underlying network environment, and uses a general congestion control algorithm model to deal with different network environments. This approach is more dependent on the input training set (historical network model), if the training set can fully cover all possible network environment and congestion Regulation algorithm, the Remy algorithm can be applied to the real network environment, the performance is very good, but if the real network and Training network difference is large, The performance of the Remy algorithm is relatively poor.

Application Scenario: The network environment is a complex heterogeneous network, hoping that the computer can automatically select the appropriate congestion control mode for different network scenarios, and requires that the existing network model can cover all possible situations.

Summarize

Each congestion control algorithm is born in a certain network environment, suitable for a particular scenario, without a once and for all algorithm. The network environment is becoming more and more complex, and the congestion control algorithm is constantly evolving. This article is not to choose a best algorithm, but simply introduces the design ideas, advantages and disadvantages of several typical algorithms, and the application scenarios, hoping to bring some inspiration to everyone.

Reference papers

[1] s.o L. Brakmo and L. Peterson. TCP vegas:new techniques for congestiondetection and avoidance. In Sigcomm, 1994. Proceedings. 1994 Internationalconference on. ACM, 1994.

[2] V.jacobson, "congestion avoidance and control," in ACM sigcomm computercommunication Review, vol. 18. ACM, 1988, pp. 314–329.

[3] L. X. I. R. Sangtae Ha. Cubic:a new Tcp-friendlyhigh-speed TCP Variant. In Sigops-osr, July 2008. ACM, 2008.

[4] C.s G. S. H. Neal Cardwell, Yuchung Cheng and V. Jacobson. bbr:congestion-based congestion control. ACM Queue, 14 (5): 20{53, 2016.

[5] K.winstein and H. Balakrishnan. TCP Ex machina:computer-generated Congestion control.in Proceedings of the ACM Sigcomm, 2013.

Discussion on TCP congestion control algorithm