Linux Network namespace-single-host self-ring test and Policy Routing

Some manufacturers went to the company for tests last week and obtained a network processing-accelerated PCIe card, which is said to be very powerful. What impressed us the most was the 40 thousand m optical port, so I can't wait to see the invisible shock of optical port forwarding.
However, how can I test it with only four optical ports? At least you need a "peer! No one should want to carry three machines to test the forwarding performance between customers. Of course, you cannot expect the customer to have the "peer" device you need, for example, our company does not have this kind of device to connect to the 10-ge optical port, but it happened that there was a device with a 10-ge optical port that day, but it happened. The best test method is of course independent of any external device. The obvious method is to do self-ring.
The twisted pair wires at the RJ45 port can be used as the physical layer self-ring, which can be 1/3 or 2/6 transient, so that one network card of a machine can be both sent and received by itself, but what can you do to compare your hair with a slightly coarse fiber? The actual solution is of course solved by software. on Linux, you can use netns, that is, net namespace.
Netns is a very interesting thing. It allows you to simulate multiple network devices on one machine. The meaning of this is different:
1. netns can be used to make full use of idle processor resources, especially when your performance of multiple network adapters is less than the CPU;
2. netns can be used to isolate different types of network applications and implement different policies for each category;
3. Using netns means virtualization, but it is more flexible than virtual machines.
A net namespace has its own route table, iptables policy, and device management mechanism. It is completely isolated from other netns. For example, if you add eth0 to netns1, the applications in netns2, nic device management is only an element in netns, and there are many other elements. For example, the iptables policy you configured in netns1 has no impact on packets in netns2. In short, if you understand the Linux kernel source code, as long as the structures with the net struct fields, such as skb and net_device, are attached to netns.
So how should I create a self-ring? My device has four NICs, and I want to communicate with each other through 2 and 3. Its logical topology is as follows:
PC1/eth0 ---- PC2/eth1 (forward) PC2/eth2 ---- PC3/eth3
Set eth0 and eth3 in two different netns, connect eth0 and eth1 with cables, connect eth2 and eth3, and set the IP addresses of eth0 and eth1 in one CIDR block, set the IP addresses of eth2 and eth3 to another CIDR block. What should I do if I do not practice fake tricks? It is also simple:
1. Add two netns
Ip netns add t1
Ip netns add t2
2. Add eth0 to t1 and set the IP address
Ip link set eth0 netns t1
At this time, ifconfig will not see eth0, and you will not even be able to see eth0 even when you execute ls/sys/class/net. Only the ip netns exec t1 ls/sys/class/net can be seen.
Ip netns exec t1 ifconfig eth0 192.168.1.200/24
3. Add eth3 to t2 and set the IP address
Ip link set eth3 netns t2
At this time, ifconfig will not see eth3, and you will not even see eth3 even if you execute ls/sys/class/net. You can only see eth3 by executing ip netns exec t2 ls/sys/class/net.
Ip netns exec t1 ifconfig eth3 172.16.1.200/24
4. Set eth1 and eth2 addresses
Ifconfig eth1 192.168.1.1/24
Ifconfig eth2 172.16.1.1/24
5. Set the default routes for two netns
Ip netns exec t1 route add default gw 192.168.1.1
Ip netns exec t2 route add default gw 172.16.1.1
6. Test
Ping eth3 address in netns t2 in netns t1
Ip netns exec t1 ping 172.16.1.200
After the preceding configuration, packets sent from eth0 will be sent to eth1 through the network cable (instead of loopback in the local route table), and then sent from eth2 through eth1 forward. Received through the network cable to the destination eth3 cup. In the whole process, there is one machine, and the effect is like three machines. With this mechanism, do you no longer have to worry about building a test environment?
In addition to the self-loop test, netns can also be used to set policy routes, which do not require ip rule. Imagine a scenario where you run both programs P1 and P2. The local LAN has two egress ports to reach the Internet. You want P1 to communicate with the outside world through gw1, p2 communicates with the outside world through gw2. The constraint is that your machine only has one Nic eth0. What should I do? You can use iptables to mark packets P1 and P2, and then use ip rule to set a policy route. In addition, you can use setsockopt directly in P1 and P2 applications to set ipmark, iptables is not required. However, this is all outdated. In 2014, I needed a different method.
I don't know how to express my thinking process, but it is simple to give an operation sequence, because doing so can indeed meet the needs, then the person who saw this article pushed back according to the operation steps to get a thought process. First of all, you need to understand that the Linux kernel supports a virtual network card type, namely, veth. Generally, veth is paired, and packets sent from a veth can directly reach its peer veth, if you are interested, you can check the Linux kernel drivers/net/veth. c, and drivers/net/tun. c is no different. It's simpler. The first step is to create a pair of veth:
Ip link add veth1 type veth peer name veth2
In this case, two NICs are added besides eth0. All NICs are lo, eth0, veth1, and veth2. There is a fact in the middle, that is, there is a virtual link between veth1 and veth2 to connect the two NICs, just like two physical NICs connected by a twisted pair. I want P1 packets to be sent through veth1 and then to veth2. But how can I send the packets to the physical line through eth0? It's too simple. Use bridge:
Brctl addbr br0
Brctl addif br0 eth0 veth2
At the same time, the LAN where veth1 and br0 are located is set in an IP network segment, so that all the sub-networks are connected. The logical topology of the L2 network is as follows:
Veth1 ---- veth2 (bridge) eth0 ---- gw (1, 2)
I didn't want to talk about how to set netns, but I still want to finish it because I am not playing with me for the moment. First, set veth1 to netns1 (how to create netns, no more details) and set the route:
Ip link set veth1 netns netns1
Ip netns exec netns1 route add default gw $ gw1
Route add default gw $ gw2
Is this all done? Yes, it's done. In fact, keep the default netns of br0 and there is no need to create netns2. Next, start P1 and P2:
Ip netns exec netns1 P1
P2
Okay, it's all over.
I always think that in Linux, the problem can be solved without modifying the source code, but I still like to modify the code. Why? Very simple, the source code is easy to obtain, and the source code is easy to modify. I write a lot of Netfilter extensions and made a lot of nf_conntrack modifications, even some damn socket filters... although these behaviors are self-entertaining and not applied at work, they indicate that I am not a network administrator, but a programmer. Haha, self-styled Senior Software Engineer (I still think these achievements can be applied ). However, being a skilled Network Manager is far more difficult than being a programmer. This is not the case. I once again encountered the multi-instance problem of OpenVPN. I think it cannot be done by programmers alone.
The multi-instance in the TAP mode has been ruined by the use of Linux Bridge, but the multi-instance in the TUN mode still has no perfect solution. Although the tun driver is modified, broadcast mode bonding + tun filter can be used to solve the problem, but I still think it is a method of getting angry. Therefore, even in the company, I failed to perform the entire debugging test, the result is lost. In fact, I really don't like that method. I have changed the tun IP filter scheme. It is not a standard solution. Can I use a standard method for addressing? When netns is used, the answer is yes.
Assume that two OpenVPN instances ovpn1 and ovpn2 are started on GW, and the virtual network interfaces are tun1 and tun2 respectively. In the client-connect script, ovpn2 is responsible for N1 and ovpn2 is responsible for N2. The key to the problem is: how does the data packet sent from the GW backend know whether to send the data packet to tun1 or tun2? Can this determination be performed automatically? If netns is used, it is okay. I can set two tun instances in different netns, and each netns corresponds to a veth virtual network card with the same netns, these veth peers are in another netns, so that the IP layer TUN mode virtual network card can be adapted to the Ethernet TAP mode virtual network card. Finally, Bridge these peers into a br0, so that the OpenVPN in TUN mode and the OpenVPN in TAP mode can be processed in the same way.

In any case, when you play with netns, you need to know that you are not playing with the cool and heartless virtualization operating system, nor are you really simulating two physically isolated machines, although the network of the two programs is isolated, the file system is shared. You must always prepare to use network isolation, memory usage, and file system sharing. A single machine can be used as multiple machines, and resources can be shared as one machine!

In any case, when you play with netns, you need to know that you are not playing with the cool and heartless virtualization operating system, nor are you really simulating two physically isolated machines, although the network of the two programs is isolated, the file system is shared. You must always prepare to use network isolation, memory usage, and file system sharing. A single machine can be used as multiple machines, and resources can be shared as one machine!
After understanding the above example and the final summary, I will ask how to play self-loop with a single or no Nic? This requirement may be used to test the protocol stack. The process of thinking is simple, with more layers. For example, if you have a machine with no Nic, you only need the following command to implement IP forwarding or bridge forwarding on your machine:
Ip link add v1 type veth peer name vp1
Ip link add v2 type veth peer name VP2.
Brctl addbr br0
Brctl addif csv4
Ifconfig vp1 up
Ifconfig VPNs up
Sysctl-w net. ipv4.ip _ forward = 1
Ip netns add t1
Ip netns add t2
Ip link set v1 netns t1
Ip link set v2 netns t2
Ip netns exec t1 ifconfig v1 1.1.1.1/24
Ip netns exec t2 ifconfig v2 1.1.1.2/24
Ip netns exec t1 ping 1.1.1.2
...