Use linux TC for server traffic control

The network environment of a server in the company is too high. when the server is connected to the source server for download, it runs to 400 M-500 M. in order to control the server, we studied TC. To control traffic. to control it to a small point, do not let this one occupy all the networks. TC is very powerful. many so-called hardware routes...

 

The network environment of a server in the company is too high. when the server is connected to the source server for download, it runs to 400 M-500 M. in order to control the server, we studied TC. To control traffic. to control it to a small point, do not let this one occupy all the networks. TC is very powerful. many so-called hardware routers are based on this.

TC introduction

In linux, TC has two control methods: CBQ and HTB. HTB, which are designed to replace CBQ. It is a hierarchical filtering framework.

TC includes three basic components:

Queue rules: qdisc (queueing discipline), class, and Classifiers)

Queue (queueing discipline) in TC ):

It is used to control the transmission speed of the network. through the queue, linux can Cache network data packets and smooth network traffic without interrupting connections (such as TCP) based on user settings. it should be noted that linux does not have good control over the receiving queue, so we generally only use the sending queue, that is, "sending control is not controlled ". it encapsulates the other two main TC components (class and classifier ). If the kernel needs to send packets through a network interface, it needs to add the packets to the queue according to the qdisc (queuing rule) configured for this interface. Then, the kernel extracts data packets from qdisc as much as possible and delivers them to the network adapter driver module.

The simplest QDisc is pfifo, which does not process any incoming data packets. data packets are first-in-first-out in the queue. However, it will save packets that cannot be processed by the network interface at the moment.

Queue rules include FIFO, RED, SFQ, Token Bucket, and CBQ ), CBQ is a super queue that can contain other queues (or even other CBQ ).

Class in TC

Class indicates the control policy. obviously, in many cases, we are likely to implement different traffic control policies for different IP addresses. at this time, we have to use different classes to express different control policies.

Filter rules in TC

Filters are used to insert users into specific control policies (that is, different classes ). for example, now we want to apply different control policies (A, B) to the IP addresses xxa and xxb. in this case, we can use filters to divide xxa into control policy, divide xxb into control policy B. The flags divided by the filter can be implemented by the u32 marking function or the set-mark (mostly using IPtables for marking) function of iptables.

Currently, TC can use the following filters: fwmark classifier, u32 classifier, route-based classifier, and RSVP classifier (for IPV6 and IPV4 respectively, the fwmark classifier allows us to use the Linux netfilter code to select traffic, while the u32 classifier allows us to select traffic based on the ANY header. it should be noted that filters are in QDisc and cannot act as subjects.

TC application process

Packet-> iptables (when iptables is used, iptables sets different mark based on different ip addresses)-> TC (class)-> TC (queue)

 

Application

Assume that eth0 is the server's Internet network interface.

Before you start, clear all the queue rules of eth0.

Tc qdisc del dev eth0 root 2>/dev/null

 

1) define the top-level (root) queue rule and specify the default category number

Tc qdisc add dev eth0 root handle 1: htb default 2

 

2) define the first layer's category (speed)

We wanted to define the second leaf category more, but now we can use this application.

Tc class add dev eth0 parent classid htb rate 98 mbit ceil 100 mbit prio 2

Tc class add dev eth0 parent classid htb rate 1 mbit ceil 2 mbit prio 2

Note: The above is the speed at which we control the output server. one is 98 M and the other is 2 M.

Rate: The bandwidth value guaranteed by a class. if there are more than one class, make sure that the sum of all subclasses is less than or equal to the parent class.

Prio: used to indicate the competitiveness of borrowed bandwidth. the smaller the prio, the higher the priority, and the stronger the competitiveness.

Ceil: ceil is the maximum bandwidth value of a class.

At the same time, in order not to allow a session to occupy the bandwidth forever, add a fair queue sfq.

Tc qdisc add dev eth0 parent handle 2: sfq perturb 10

Tc qdisc add dev eth0 parent handle 3: sfq perturb 10

 

3) set the filter

The filter can use its own u32 or iptables to mark

Specify to filter 192 .. 168.0.2 in The root class 1:0 and use the rule to give it a 98 M speed. The statement is as follows:

Tc filter add dev eth0 protocol ip parent 1:0 u32 match ip src 192.168.0.2 flowid

Tc filter add dev eth0 protocol ip parent 1:0 u32 match ip src 192.168.0.1 flowid

For all ip addresses

Tc filter add dev eth0 protocol ip parent 1: prio 50 u32 match ip dst 0.0.0.0/0 flowid :10

 

Use Iptables with filters

You can also use this method, but you need to use the following iptables command to mark

Tc filter add dev eth0 parent 1: protocol ip prio 1 handle 2 fw flowid

Tc filter add dev eth0 parent 1: protocol ip prio 1 handle 2 fw flowid :3

Iptables only needs to be marked.

Iptables-t mangle-a postrouting-d 192.168.0.2-j MARK -- set-mark 10

Iptables-t mangle-a postrouting-d 192.168.0.3-j MARK -- set-mark 20

 

TC controls the highest speed

Rate ceiling speed limit

The ceil parameter specifies the maximum bandwidth that a class can use to limit the bandwidth that the class can borrow. the default ceil is the same as the rate.

This feature is useful to ISPs because they generally limit the total number of users to be served, even if other users do not request services. (ISPS really wants users to pay more for better services). Note that the root class is not allowed to be borrowed, so ceil is not specified.

Note: the value of ceil should be at least as high as the speed of its class, that is, ceil should be at least as high as any of its subclass.

Burst

Network hardware can only send one packet at a time, which only depends on the speed of one hardware. the link sharing software can use this capability to dynamically generate multiple connections running at different speeds. therefore, the speed and ceil are not an instant measurement but an average value of the packet sent in a time period. the actual situation is how to make a class with a small traffic at a certain time to provide the class to other classes at the maximum rate. the parameters of burst and cburst control how much data can be sent to other classes at the maximum hardware speed without any effort.

If the cburst is smaller than a theoretical packet, the burst will not exceed the ceil rate. The same is true for the same method of TBF.

You may ask why you need bursts. because it can easily increase the forwarding speed on a very crowded link. for example, WWW traffic is abrupt. visit the home page. sudden Acquisition and reading. in the idle time, burst will "charge" again.

Note: the values of burst and cburst must be at least as large as those of its subclass.

TC command format:

Join

Tc qdisc [add | change | replace | link] dev DEV [parent qdisc-id | root] [handle qdisc-id] qdisc [qdisc specific parameters]

Tc class [add | change | replace] dev DEV parent qdisc-id [classid class-id] qdisc [qdisc specific parameters]

Tc filter [add | change | replace] dev DEV [parent qdisc-id | root] protocol prio priority filtertype [filtertype specific parameters] flowid flow-id

Display

Tc [-s |-d] qdisc show [dev DEV]

Tc [-s |-d] class show dev DEV tc filter show dev DEV

View TC status

Tc-s-d qdisc show dev eth0

Tc-s-d class show dev eth0

Delete tc rules

Tc qdisc del dev eth0 root

Instance

 

Use TC download to limit the speed of a single IP address

Tc qdisc add dev eth0 root handle 1: htb r2q 1

Tc class add dev eth0 parent 1: classid 1:1 htb rate 30 mbit ceil 60 mbit

Tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.1.2 flowid 1:1

The download speed of 192.168.1.2 can be limited to 30 Mbit and up to 60 Mbit. r2q indicates that the entire network bandwidth is not limited because no default root is available.

Use TC to control the speed of the entire IP address segment

Tc qdisc add dev eth0 root handle 1: htb r2q 1

Tc class add dev eth0 parent 1: classid htb rate 50 mbit ceil 1000 mbit

Tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.111.0/24 flowid 1:1

The bandwidth between 192.168.111.0 and 255 can be limited to kB, and the actual download speed is about KB.

In this case, all machines in the CIDR block share the bandwidth of KB.

You can also add an sfq (random fair queue)

Tc qdisc add dev eth0 root handle 1: htb r2q 1

Tc class add dev eth0 parent 1: classid htb rate 3000 kbit burst 10 k

Tc qdisc add dev eth0 parent 1:1 handle 10: sfq perturb 10

Tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip dst 192.168.111.168 flowid 1:1

Sfq can prevent an ip address in a segment from occupying the entire bandwidth.

Using TC to control the external speed of the server is 10 MB

As shown in the following figure, I want to manage a server and only send 10 MB of data to external servers.

Tc qdisc del dev eth0 root

Tc qdisc add dev eth0 root handle 1: htb

Tc class add dev eth0 parent 1: classid htb rate 100 mbit ceil 100 mbit

Tc class add dev eth0 parent classid htb rate 10 mbit ceil 10 mbit

Tc qdisc add dev eth0 parent :10 sfq perturb 10

Tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dst 220.181.xxx.xx/32 flowid 1:1

# The above section sets the default value for the 220.181.xxx.xx/32 instance, mainly to prevent the ip connection from being controlled.

Tc filter add dev eth0 protocol ip parent 1: prio 50 u32 match ip dst 0.0.0.0/0 flowid :10

# By default, all traffic passes through this