Simulate the network packet sending and receiving delay for the specified IP port on Linux.

Simulate the network packet sending and receiving delay for the specified IP port on Linux.

When compiling network applications, we usually test and debug the network on a LAN or even a local machine. Is there a way to simulate a complex Internet environment-especially network latency-for applications without modifying their own code in an Intranet environment with good network conditions? This was my idea when I was writing network programs at school, but I did not study it carefully until recently I was writing cross-server code in my company.

Cross-server communication involves communication between multiple servers and between servers and clients. The process is complex. In each step, you must correctly handle network exceptions, delays, and disconnections. It is extremely troublesome for testers to test network latency by modifying code or using a breakpoint, and there are limited latency cases that can be simulated. Therefore, if a third-party tool simulates network latency for a socket (IP port) used by the application, the tester should like it very much.

The tool originally found includes the tc Command provided by Linux (which must be used with the module netem provided by tc) and a third-party tool dummynet. The former concept is complex, and the command line parameters are also complex. The latter is cross-platform and available on Windows. However, it is very troublesome to install it on Linux. To compile the kernel module provided by dummynet, we need to compile the source code of the Linux kernel in the correct version. In the end, we decided to use tc. The plan is to use tc to set the network delay of packets receiving and sending respectively for the server port, which can solve the problem that tc can only work in Linux. The tc manual and many articles on the Internet mentioned that tc can only set the packet sending delay, but not the package receiving delay. However, as long as the Linux built-in ifb (Intermediate Functional Block device) kernel module and a little trick, tc can set the package delay.

In fact, tc can easily set network latency for a network card:

# Tc qdisc add dev wlan0 root netem delay 1 s

This command sets the network latency of 1 second for the packet sent by the wireless network adapter wlan0.

You can ping other machines in the LAN for verification:

# Ping-c 4 192.168.1.5

PING 192.168.1.5 (192.168.1.5) 56 (84) bytes of data.

64 bytes from 192.168.1.5: icmp_req = 1 ttl = 64 time = 1002 MS

64 bytes from 192.168.1.5: icmp_req = 2 ttl = 64 time = 1001 MS

64 bytes from 192.168.1.5: icmp_req = 3 ttl = 64 time = 1002 MS

64 bytes from 192.168.1.5: icmp_req = 4 ttl = 64 time = 1004 MS

--- 192.168.1.5 ping statistics ---

4 packets transmitted, 4 bytes ed, 0% packet loss, time 3006 MS

Rtt min/avg/max/mdev = 1001.446/1002.830/1004.967/1.642 MS

However, this will affect all packets sent through this Nic. This is not what I want. I just want to set network latency for the specified port on the server, and do not want to affect other ports, so I should remove this delay rule:

# Tc qdisc del dev wlan0 root

To set latency only for the specified IP port, we need to use three complex concepts in tc: qdisc (queuing rules), class (class), and filter (filter ). It took me many days to understand how they work together. I don't want to explain these concepts in detail here (For details, refer to the references listed at the end of this Article), but write down how I did it.

Assume that two communication applications are running on the local machine, program A listens on port 14100, and program B establishes A TCP connection between port 14100 of program. I want to set the latency in the communication direction from B to A by setting qdisc and filter on the sender end of the local back-to-network interface card to filter all packets sent to the local port 14100, set latency for these packages.

First, add a root qdisc in the local logoback Nic lo:

# Tc qdisc add dev lo root handle 1: prio bands 4

This qdisc has four classes and the handle id is 1 :. If no filter exists, the packets received by tc from the IP protocol layer will enter 1st ~ according to the TOS (Type of Service) field of the IP header ~ 3rd classes (same as the pfifo_fast rule), 4th classes are useless. Now add a qdisc with a latency of 5 seconds to the 4th classes:

# Tc qdisc add dev lo parent 1:4 handle 40: netemDelay 5S

Add a filter to the root qdisc and send all the packets sent to port 14100 to the 4th class:

# Tc filter add dev lo protocol ip parent 1:0 prio 4 u32 \

Match ipDport 141000 xffff flowid

In this way, you can.

If you want to cancel the network delay, you can delete the filter. First, list the filter information:

# Tc-s filter show dev lo

Filter parent 1: protocol ip pref 4 u32

Filter parent 1: protocol ip pref 4 u32 fh 800: ht divisor 1

Filter parent 1: protocol ip pref 4 u32 fh 800: 800 order 2048 key ht 800bkt 0 flowid (rule hit 672 success 76)

Match 00003714/0000 ffff at 20 (success 76)

The above information shows that 76 packages are filtered out by filters. These packages are all sent to port 14100 by the local loopback Nic. Delete filter now:

# Tc filter del dev lo pref 4

However, the preceding scenario is that both applications are local. Therefore, you can set the sending end of the ring back Nic to control the packet receiving speed of port 14100 (on the ring back Nic) in disguise. If program B is on another machine, ifb is required. Ifb will open a virtual network card in the system. If we redirect the packet received by wlan0 (the actual Nic) to ifb, ifb will send the packet to wlan0 again, and finally send it to the IP layer through wlan0. The upper-layer protocol has no idea. Therefore, you can set the packet sending delay of ifb to achieve the packet receiving delay of wlan0.

To use ifb, you must first load the ifb kernel module, which is included in Debian 7:

# Modprobe ifb

Through the ip command, we can see that ifb0 and ifb1 are added to the system:

# Ip link list

1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc prio state UNKNOWN mode DEFAULT

Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00

2: eth0: <NO-CARRIER, BROADCAST, MULTICAST, UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000

Link/ether 60: eb: 69: 99: 66: 54 brd ff: ff

3: wlan0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc mqstate UP mode DORMANT qlen 1000

Link/ether 1c: 65: 9d: a9: db: 01 brd ff: ff

10:Ifb0: <BROADCAST, NOARP> mtu 1500 qdisc noopState DOWNMode DEFAULT qlen 32

Link/ether 7a: 2a: 33: 6b: e7: f7 brd ff: ff

11:Ifb1: <BROADCAST, NOARP> mtu 1500 qdisc noopState DOWNMode DEFAULT qlen 32

Link/ether ce: ba: f4: 38: df: 6c brd ff: ff

Start ifb0 NIC:

# Ip link set ifb0 up

Make sure that the ifb0 Nic is Enabled:

# Ip link list

1: lo: <LOOPBACK, UP, LOWER_UP> mtu 16436 qdisc prio state UNKNOWN mode DEFAULT

Link/loopback 00: 00: 00: 00: 00: 00 brd 00: 00: 00: 00: 00: 00

2: eth0: <NO-CARRIER, BROADCAST, MULTICAST, UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000

Link/ether 60: eb: 69: 99: 66: 54 brd ff: ff

3: wlan0: <BROADCAST, MULTICAST, UP, LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT qlen 1000

Link/ether 1c: 65: 9d: a9: db: 01 brd ff: ff

10:Ifb0: <BROADCAST, NOARP, UP, LOWER_UP> mtu 1500 qdisc pfifo_fastState UNKNOWNMode DEFAULT qlen 32

Link/ether 7a: 2a: 33: 6b: e7: f7 brd ff: ff

11: ifb1: <BROADCAST, NOARP> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 32

Link/ether ce: ba: f4: 38: df: 6c brd ff: ff

Add ingress qdisc in wlan0, that is, the queuing rule for packet collection:

# Tc qdisc add dev wlan0 ingress

Redirect packets received by wlan0 to ifb0:

# Tc filter add dev wlan0 parent ffff :\

Protocol ip u32 match u32 0 0 flowid 1:1 action mirred egressRedirect dev ifb0

Next, set qdisc and filter on the ifb0 sender, and set a latency of 5 seconds for packets sent to port 14100 as before:

# Tc qdisc add dev ifb0 root handle 1: prio bands 4

# Tc qdisc add dev ifb0 parent :4 handle 40: netemDelay 5S

# Tc filter add dev ifb0 protocol ip parent 1:0 prio 4 u32 \

Match ipDport 141000 xffff flowid

Success! From start to end, the entire process did not make any changes to the application itself, nor did it change the network protocol, nor did it affect other running programs on the machine.

However, these tc commands are still too complex for testers. After all, tc's target users seem to be professional network administrators and system administrators. Originally, I only wanted to simulate network latency, but I did not expect to find that this involves a very big topic-traffic control Orz. Then I will encapsulate them into simple commands.

References