Linux TC htb queue-based traffic management example

It should also be repeated that the TC rule involves three items: "queue classifier (class) filter. In addition, it is not controlled by the Control Service. Today, we will demonstrate how TC uses the set-Mark Function of filter to control traffic in the htb queue mode. Many people think that htb is much easier to operate than CBQ, so do I.

Suppose the environment: There are two NICs in Linux. One is eth1 and the other is eth0 Is intranet. htb is performed on eth0. (Note: The filter flag can be used for u32 marking or the set-Mark Function of iptables. If iptables is used for marking, the downlink speed is controlled at eth0, however, tagging should be performed before entering eth0, so you must set "-I eth1" in the firewall"

 

The main command is as follows:

 

1. Create an htb Root

# TC qdisc add Dev eth0 root handle 1: htb default 20

2. Create an htb class. The traffic limit is restricted here and an emergency is set.

# TC class add Dev eth0 parent 1: classid htb rate 200 kbit Ceil 200 kbit burst 20 K (Burst Traffic)

3. Create a filter rule to filter out the data to limit the traffic and send it to the above class to limit the speed.

# TC filter add Dev eth0 parent 1: PRIO 1 Protocol IP u32 Match ip sport 80 0 xfff flowid 1:1

Note: Keep interactive data packets with a low latency and obtain idle bandwidth first, for example:

SSH Telnet DNS quake3 irc ftp controls SMTP commands and syn-tagged data packets. In order to ensure that the upstream data stream does not harm the downstream stream, the ACK packet must be placed before the queue, because the ACK of the downstream data must compete with the same row of stream.

 

TC + iptables + htb + SFQ

1. the TCP/IP protocol specifies that each packet must receive a acknowledge message. That is to say, a reply to the received message is required for the transmitted data to determine the subsequent transmission speed, and decide whether to re-transmit the lost data. Part of the uplink bandwidth is used to transmit the ACK data. when the uplink bandwidth usage is high, the ACK data transmission speed is affected, and the download speed is also affected,

2. The test proves that when the upload is full, the download speed becomes 40% or even lower, because the Upload File (including FTP upload and email SMTP) is large, A single traffic volume makes the bandwidth overhead packets and all data packets queue and wait according to the first-in-first-out principle, this explains why the entire network speed becomes slow when someone uploads a file or sends a large email over FTP.

Solution speed:

To solve these speed problems, the data passing through the line is distributed in a regular manner. by transferring the bandwidth bottleneck to our Linux router, we can control the bandwidth a little less than the bandwidth we purchased. in this way, we can easily use TC technology to distribute and control the passed data.

Our imagination is like a driveway on the road. There are high-speed lanes, small lanes, large lanes, and high-speed syn ack icmp ssh, FTP-data and SMTP, which require a large amount of transmission, cannot block the entire road.

In Linux, the TC (Traffic Control) has such a role, as long as the control is appropriate, it will certainly have a significant effect. Combining TC and iptables is the best method.

We set a filter to use iptables to classify data packets. Because iptables is more flexible and can set counters for each rule, iptables uses the mangle chain to mark data packets and tells the kernel, the data packet has a specific fwmark mark value (handle x fw) indicating that it should be sent to that class (classid X: X), and PRIO is the priority value, indicates that important data should first pass through that channel. First, select Queue (htb ),

Generally, the first-in-first-out queue of the first-in-first-out queue is used by default, that is, the packet is first-in-first-out, the subsequent package can only be sent after the previous package is sent, so that even if it is a small ack package, it will wait, so that the upload will affect the download, even if you have a large download bandwidth, you can't do anything about it.

Htb (hierarchical token bucket, layered token bucket), which works like CBQ, but does not rely on calculation of idle time. It is a classification token bucket filter ., it has few parameters.

Structure Diagram: 1:

~~~~~~~~~~~~~~~~ '~~~~~

~ _________ 1:1 ~~~~~~~~~ ________

| ~~~ | ~~~~ | ~~~~ | ~~~~~ | ~~~~~~~~ | ~~~~~~~~ | ~~~~~~~ |

~~~ ~~~~~~~~~~~~ ~~~ ~~~ ~~ 1: 24

Priority:

Based on the above example, start the script:

Parameter description:

Rate: The bandwidth value guaranteed by a class. If there are more than one class, make sure that the sum of all subclasses is less than or equal to the parent class,

Ceil: Ceil is the maximum bandwidth value of a class.

Prio: it is the priority setting. The greater the value, the smaller the priority. If it is to allocate the remaining bandwidth, the smaller the value will give priority to the remaining idle bandwidth.

Generally, it is about 50%-80% for big data, and ceil is recommended to not exceed 85% to avoid a session occupying too much bandwidth.

 

Rate can be allocated according to various types of requirements:

Uplink uplink 320 k

Dev = "ppp0"

Uplink ink = 300

Downlink = 1500 downlink 3200k is about half of the total, so as to get more disconnect connections.

　　

1. Add a root queue. If no data packet is classified, is the default class:

TC qdisc add Dev $ Dev parent 1: htb default 24

1.1 Add a master class1 under the root team: the speed is 300 K

TC cladd add Dev $ Dev parent 1: classid htb rate 300 kbit Ceil 300 kbit PRIO 0

1.1.1 create the first leaf class under trunk Class 1. This is the highest priority class, which requires a high-priority and high-speed packet to take this channel, such as syn ack icmp.

TC class add Dev $ Dev parent classid htb rate 300 kbit Ceil 300 kbit PRIO 1

Is a very small and most important data packet channel. Of course, it must be divided into multiple vertices, or even occupied first if necessary, but it generally does not. Therefore, full speed is given.

1.1.2 create the second leaf category under main Class 1, which is a next-highest priority class, such as our important CRM data.

TC class add Dev $ Dev parent classid htb rate 300-150kbit Ceil 300-50kbit PRIO 2

Is a very important data path. You can give at least half of the data to multiple points, but you can add more points when necessary.

Rate planning = + + + is generally around 50%-80%.

1.2 create a secondary stem class classid under the root class. All the following priorities of the primary class are lower than those of the primary class to prevent important data congestion.

TC class add Dev $ Dev parent 1: classid htb rate 300-150kbit PRIO 3

1.2.1 create the first leaf class under the secondary stem class and run such as HTTP and pop.

TC class add Dev $ Dev parent classid htb rate 100 kbit Ceil 300-150kbit PRIO 4

HTTP, pop is the most commonly used. For the sake of too many people, blocking is caused, and we cannot give too much or too little.

1.2.2 create a second leaf category under the secondary stem category. Do not set the speed too high to prevent large attachments from occupying a large amount of bandwidth, such as SMTP.

TC class add Dev $ Dev parent classid htb rate 30 kbit Ceil 300-160kbit PRIO

I plan to use it for SMTP. The priority is lower than to prevent large attachments from occupying a large amount of bandwidth.

1.2.3 set up the third leaf category under the secondary stem category. Do not use too much bandwidth to prevent network congestion caused by a large amount of data, such as ftp-data.

TC class add Dev $ Dev parent classid htb rate 15 kbit Ceil 300-170kbit PRIO 6

I plan to give FTP-Data. Like, it is very likely that a large number of files will be uploaded. Therefore, the rate cannot be too large, while other files can be larger when there is surplus, ceil settings are larger.

1.2.4 create the fourth leaf category under the secondary stem category. There is no need for too much bandwidth for indifferent data channels to prevent indifferent people from interfering with the business.

TC class add Dev $ Dev parent classid htb rate 5 kbit Ceil 300-250kbit PRIO 7

Is an indifferent channel, which is generally not the channel we usually need at work, give small points to prevent these people from interfering with normal work needs.

Add another queue rule under each category. The random fair queue (SFQ) is not used by a connection to ensure the average fair use of bandwidth.

# SFQ (Stochastic fairness queueing random fair queue), the keyword of SFQ is "session" (or stream), mainly for a TCP session or UDP stream, traffic is divided into a considerable number of FIFO queues. Each queue corresponds to a session.

Data is sent in a simple round-robin manner, and each session receives sending opportunities in order. This method is fair, ensuring that each session is not overwhelmed by other sessions. SFQ is called "random" because it does not really create a queue for each session, instead, a hash algorithm is used to map all sessions to a limited number of queues.

# How many seconds does the parameter perturb ([PE 'taib] Bother someone) Reconfigure the hash algorithm. The default value is 10 seconds.

TC qdisc add Dev $ Dev parent handle 111: SFQ perturb 5

TC qidsc add Dev $ Dev parent handle 112: SFQ perturb 5

TC qdisc add Dev $ Dev parent handle 121: SFQ perturb 10

TC qidsc add Dev $ Dev parent handle 122: SFQ perturb 10

TC qidsc add Dev $ Dev parent handle 123: SFQ perturb 10

TC qidsc add Dev $ Dev parent handle 124: SFQ perturb 10

Set the filter. Handle is the value of iptables as the mark, so that iptables selects different channels for different Mark values in the mangle chain, while PRIO is the priority of the filter.

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 1 handle 1 FW classid :11

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 2 handle 2 FW classid :12

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 3 handle 3 FW classid :21

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 4 handle 4 FW classid :22

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 5 handle 5 FW classid :23

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 6 handle 6 FW classid :24

######################################## ######################################## ####

Downstream restrictions:

# Set the queuing rules because some ports that often cause large file downloads are controlled to prevent them from coming too quickly, resulting in congestion and getting too fast. Directly drop, it will not waste and take up machine time and power to deal with it.

1. Set the downlink speed to around 1000-1500 K (about 50% of the bandwidth), because the speed is enough to get more concurrent download connections.

# TC qdisc add Dev $ Dev handle FFFF: Ingress

# TC filter add Dev $ Dev parent FFFF: Protocol ip prio 50 handle 8 FW police rate $ {downlink} kbit burst 10 k drop flowid: 8

If the intranet data stream is not crazy, you do not need to restrict the download. Use the # symbol to shield the above two lines.

If you want to limit the speed of any incoming data, you can use the following sentence.

TC filter add Dev $ Dev parent FFFF: Protocol ip prio 10 u32 Match ip SRC 0.0.0.0/0 police rate $ {downlink} kbit burst 10 k drop flowid: 1

################################

Start marking the data packet:

Mark mark 1-6 for different types of data packets (dport) to mark and let it go through different channels.

Mark the incoming data packet (sport) to mark 8, so that it is subject to downlink restrictions, so as not to affect the global speed too quickly.

The return method can be used to avoid traversing all the rules and speed up processing.

Set TOS processing:

Iptables-T mangle-A prerouting-m tos -- TOS minimize-delay (minimum latency)-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-m tos -- TOS minimize-delay-J return

Iptables-T mangle-A prerouting-m tos -- TOS minimize-cost (minimum cost)-J mark -- Set-mark 4

Iptables-T mangle-A prerouting-m tos -- TOS minimize-cost-J return

Iptables-T mangle-A prerouting-m tos -- TOS maximize-throughput (maximum throughput)-J mark -- Set-mark 5

Iptables-T mangle-A prerouting-m tos -- TOS maximize-througput-J return

# It is wise to increase the priority of TCP Initial connections (that is, packets with SYN.

Iptables-T mangle-A prerouting-p tcp-m tcp -- TCP-flags SYN, RST, Ack syn-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p tcp-m tcp -- TCP-flags SYN, RST, Ack syn-J return

# If you want to ping ICMP, the system has a good response. Put it in the first class.

Iptables-T mangle-A prerouting-p icmp-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p icmp-J return

# A small packet whose small packets (probably just acks) is smaller than 64 usually needs to be faster. It is generally used to confirm the TCP connection, so that it can be connected to a faster channel.

Iptables-T mangle-A prerouting-p tcp-m length -- length: 64-J mark -- Set-mark 2

Iptables-T mangle-A prerouting-p tcp-m length -- length: 64-J return

# Put FTP in the second type, because it is generally a small packet, and FTP-data in the 5th class, because it is generally a large data transfer.

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport FTP-J mark -- Set-mark 2

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport FTP-J return

Iptables-T mangle-A pretouting-p tcp-m tcp -- dport FTP-data-J mark -- Set-mark 5

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport FTP-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport FTP-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport FTP-data-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport FTP-data-J return

### Improve the priority of SSH data packets: Put it in the 1st class. You Need To Know That SSH is interactive and important, and it cannot be slowed down:

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 22-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport-J return

# SMTP mail is placed in the 4th class, because sometimes someone sends a large mail. To avoid blocking it, let it run 4th lines.

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 25-J mark -- st-mark 4

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 25-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 25-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 25-J return

# Name-Domain Server: Put it in the 1st class, so that the connection with a domain name can quickly find the corresponding address, improving the speed

Iptables-T mangle-A prerouting-p udp-m udp -- dport 53-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p udp-m udp -- dport 53-J return

### Http: put in the 3rd class, which is the most common and most commonly used

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 80-J mark -- Set-mark 3

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 80-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 80-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 80-J return

### Put POP mail in the 3rd category:

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 110-J mark -- Set-mark 3

Iptables-T mangle-A prerouting-p tcp-m tcp -- dprot 110-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 110-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 110-J return

### MICSOSOFT-SQL-SERVE: put in the 2nd class, I think it is more important here, be sure to ensure speed and priority

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 1433-J mark -- Set-mark 3

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 1433-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 1433-J mark -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 1433-J return

# Https: Put in 3rd class

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 443-J mark -- Set-mark 3

Iptables-T mangle-A prerouting-P tcpm-m tcp -- dport 443-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 443-J makr -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 443-J return

### When using VOIP, improve the performance and ensure that the voice is kept at high speed.

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 1720-J mark--set-mark 1

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 1720-J return

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 1720-J makr -- Set-mark 8

Iptables-T mangle-A prerouting-p tcp-m tcp -- Sport 1720-J return

### When a VPN is used as a VoIP service, it must take a high-speed road to avoid intermittent interruption.

Iptables-T mangle-A prerouting-p udp-m udp -- dport 7707-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p udp-m udp -- dport 7707-J return

### Put it in the 1st category, because I think objective existence is very important to me, and you can:

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport 7070-J mark -- Set-mark 1

Iptables-T mangle-A prerouting-p tcp-m tcp -- dport-J return

# Improve the priority of local data packets: Put it in class 1st

Iptables-T mangle-A output-p tcp-m tcp -- dport 22-J mark -- Set-mark 1

Iptables-T mangle-A output-p tcp-m tcp -- dport 22-J return

Iptables-T mangle-A output-p icmp-J mark -- Set-mark 1

Iptables-T mangle-A output-p icmp-J return

### Local small packet (probably just acks)

Iptables-T mangle-A output-p tcp-m length -- length: 64 -- Set-mark 2

Iptables-T mangle-A output-p tcp-m length -- length: 64-J return

######################################## #########

# After adding a mangle rule to pretoutrin, use this rule to end the prerouting table: that is to say, it is unnecessary to hand over packets that have not been marked before to for actual processing, because is the default class, but it is still marked to maintain the consistency of the entire setting, and in this way, the packet count of the rule can be seen:

Iptables-T mangle-A prerouting-I $ Dev-J mark -- Set-mark 6

### Limits on a person: iptables-T mangle-I prerouting 1-s 192.168.xx.xx-J makr -- Set-mark 6

### Iptables-T mangle-I prerouting 2-s 192.168.xx.xx-J return

######################################## ###########

U32 applications:

TC filter add Dev eth0 parent 1:0 Protocol ip prio 1 u32... this is the so-called u32 match, which can match any part of the data packet.

Based on the source/destination address:

Match ip SRC 0.0.0.0/0

Match ip DST 1.2.3.0/24

A single IP address can be expressed as/32.

 

It can be expressed as follows based on the source/destination port:

Match ip sport 80 0 xFFFF

Match ip dport 80 0 xFFFF

 

According to the IP protocol:

Match IP protocol (udp tcp icmp gre IPSec)

For example, the ICMP protocol is 1 match IP protocol 1 0xff

Example:

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 1 u32 Match ip DST 4.3.2.1/32 flowid 10:1

TC filter add Dev $ Dev parent 1:0 Protocol ip prio 1 u32 Match ip SRC 4.3.2.1/32 Match ip sport 80 0 xFFFF flowid 10:1

From: http://www.netren.org/index.php/linux-tc/44-tc-filter.html