Linux Traffic Control

In Linux, IP commands implement part of the functions implemented by most professional routers. The other command TC implements almost another function on professional routers, the remaining functions are all-inclusive. If you run another zebra, the Linux host will be an advanced router, and its speed will no longer depend on the software, however, it depends on your hardware. Although a complete Linux operating system on a 386 machine is completely dirty, it cannot be thrown to the public network for use as a real router. By the way, "iptables guide 1.1.19" and "Linux's advanced routing and traffic control howto" are two very comprehensive, interesting, and easy to use documents.
In Linux, traffic control for egress is easy to implement, and its implementation is quite good. In essence, recursive [queuing rules, filters, and classification queues] are used, the queuing rules of the classified queue are classified into the next level of three tuples. Data first goes through the top queuing rules, and then uses its filter to classify data packets into one category, during this period, data packets can be placed into a queue of this type, if there is a queue under this category. This is the meaning of recursion. With these three tuples, you can easily configure Complex Traffic Control Policies at the egress.
In addition to throttling at the egress, a dedicated traffic control system using a bridge can also be very good. The traffic control of a bridge is much less affected than a router, after all, it almost works at the link layer, and the speed must be very fast.
Throttling cannot independently control the entry rate, but can force the entry rate limit because the entry is not as complex as the exit queue. Essentially, there is no real queue for the entry, and the ingress of Tc for the entry does not use the queue. It uses netfilter to implement the classification framework of TC. We can't see any queuing operations in the ingress_enqueue function. We only call tc_classify to make a decision, and then return the results of nf_xxx based on this decision. Obviously, to make such a decision, some guidance information must be provided. The guidance information is actually provided by the netfilter framework. It should be given before the hook of ing_ops is called. It uses the mangle table, we can see that the hook and ing_ops of the mangle table are both mounted on the prerouting, but the priority of the mangle is relatively high. It can mark the data packet before the ing_ops takes over the data packet, this tag serves as the guiding information for ingress to make a choice. As described in ingress_enqueue, it does not queue data packets, but simply throws or discards data packets (based entirely on the netfilter method ), without a queue, there is no cache and no queue. In terms of operations, we cannot mount more queuing rules and filters to the root user. In essence, recursive filters and classifiers are mainly used for final debugging. The purpose of scheduling is to control the data rate sent to the network. We do not need to do this for the incoming traffic, because at the prerouting point, the following operations are routing, for forward and local-in, the next operation is to transmit data to the NIC and process, there is no need to do the so-called scheduling, so for ingress, you can only set the upper limit easily, but cannot set the speed limit independently. You must use the netfilter mechanism. The Entry Traffic Monitoring in the kernel is completed through the police module. Like most kernel data structures, it is also a complex struct with many built-in callback functions. A struct is very important, it is struct tcf_police, which contains intuitive fields such as burst, MTU, and rate. In its initialization function tcf_act_effece_locate, these fields are set according to the user space configuration. Then, a timer is started to process statistics in the callback function when the timer expires to calculate the current traffic, in the processing function tcf_act_police with incoming traffic, statistics are updated (provided for the timer). Based on the calculation result of the timer, you can decide how to handle the data packet. If the data packet exceeds the specified rate or exceeds the threshold, the rule is determined according to the configured action.
The overall action is actually that data flows through the prerouting chain of Netfilter and goes through mangle and ingress in sequence. mangle tags data packets based on iptables configuration, then ingress will call the qdisc team-up operation on it later. The latter calls tc_classfiy, then traverses all the policies, and finally reaches the police module, and the traffic is monitored.
The essence of this implementation is that we cannot control when the host outside the router sends data at what rate, and what we can control is when we send data at what rate, the principle behind this is that we cannot control others, but we can control ourselves. In addition, how can I add a queue to the ingress traffic control? For example, if you do not use netfilter, but add a queue at the NIC level-like egress traffic control, what should you do? In general, this is not good, because the data is sent by others after all. If the speed limit is implemented in this way, the flow control is purely for speed limit, which does not play a major role in bandwidth savings, just like a reservoir, it is generally used for flood relief at the opening or closing of the reservoir to store water. If this is not the case, but the flood is restricted at the entrance of the reservoir, Flood Resistance will be achieved in.
Configuration instance (changed from "Linux advanced routing and traffic control"):
Outlet speed limit:
TC qdisc add Dev eth1 root handle 1: CBQ avpkt 1000 bandwidth 100 Mbit
TC class add Dev eth1 parent 1: classid CBQ rate 512 kbit allot 1500 PRIO 5 bounded isolated
TC filter add Dev eth1 parent 1: Protocol ip prio 16 u32 Match ip DST 6.6.6.6 flowid 1:1
Entry speed limit:
Iptables-A prerouting-I eth0-T mangle-P TCP-J mark -- Set-mark 1
TC qdisc add Dev eth0 handle FFFF: Ingress
TC filter add Dev eth0 parent FFFF: Protocol ip prio 50 handle 1 FW police rate 512 kbit burst 10000 MTU 9 K drop flowid: 1