Docker Advanced Networking Practices Linux Network namespace & pipework

Objective

In the previous article, "to understand the basic principles of the Docker network", we have learned about the 4 kinds of drivers that Libnetwrok offers in Docker, but there are some limitations in each approach. Assuming you need to operate a data center network, we have a host of hosts, thousands of Docker containers running on each host, and what happens if you use 4 kinds of network drivers, let's analyze:

1. Using the host driver allows the container to share the same network stack with the host, which seems to solve the network problem, but it does not actually use the isolation of networks namespace and lacks security.
2. Using Docker's default bridge driver, the container has no external IP and can only communicate externally through NAT. This approach does not solve the problem of direct communication across host containers, and it is difficult to meet the business requirements under complex scenarios.
3. The overlay driver can be used to support cross-host network communication, but must be configured and used with Swarm for cross-host network communication.
4. Using a NULL driver does not actually make any network configuration.

It can be seen that in order to realize the cross-host network communication between a large number of containers in the data center, in order to more flexibly realize the sharing and isolation of the inter-container network, and in order to make the network configuration more automated when managing thousands of containers, we need to understand more advanced network scenarios.

? This article and later article will use some tools and additional operations to break through the existing limitations of the Docker network, to achieve some of the more advanced features to meet the complex needs of practical applications.

Play the Linux network namespace.

? The Linux network namespace has been described in the previous article, and in this article we will learn from a practical perspective how to operate the Linux network namespace under a Linux system.

? IP is a powerful network configuration tool in Linux system, it can not only replace some traditional network management tools, such as ifconfig, route, etc., but also can achieve richer functions. The following describes how to use IP commands to manage network namespace.

Use IP netns to operate network namespace

The IP netns command is used to operate the network namespace instructions, as follows.

• Create a network namespace:

  #创建一个名为net-test的network namespace[[email protected] ~]# ip netns add net-test

• List network namespace that already exist in the system:

  [[email protected] ~]# ip netns lsnet-test

• Delete a network namespace:

  [[email protected] ~]# ip netns delete net-test

• Execute a command in the network namespace:

#命令格式
IP netns exec <network nameapce name> <command>

#比如显示net-test Namespace network card information, routing information

[[email protected] ~]# ip netns exec net-test ip addr1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN qlen 1     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 [[email protected] ~]# ip netns exec net-test route -n Kernel IP routing table Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

In fact, if you think the IP netns exec to execute the command is troublesome, you can also use to start a shell to match:
#命令格式
IP netns exec <network nameapce name> bash

In this way, the command can be executed as if the user had entered the network namespace, and if you want to exit the bash, enter exit.

Configuring network adapters with IP for network namespace

When you create a network namespace using the IP netns add command, you have a separate cyberspace that can be configured according to your needs, such as adding network cards, configuring IP, setting up routes, and so on. The following is an example of how to do this, using the previously established network namespace named Net-test.

? When creating a network namespace using the IP command, a loopback device (loopback Interface:lo) is created by default. The device does not start by default and it is best to start it.

[[email protected] ~]# ip netns exec net-test ip link set dev lo up

Create two virtual network cards veth-1 and veth-2 on the host:

[[email protected] ~]# ip link add veth-1 type veth peer name veth-2

Add the veth-2 device to the Net-test network namespace, and the veth-1 stay on the host:

[[email protected] ~]# ip link set veth-2 netns net-test

Now net-test This network namespace has two network cards (LO and veth-2), verify the look:

[[email protected] ~]# ip netns exec net-test ip link1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00222: [email protected]: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000    link/ether 92:24:fd:44:c6:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0

Next, you can assign IP to the NIC and start the network card:

#在主机上为veth-1配置IP并启动[[email protected] ~]# ip addr add 10.0.0.1/24 dev veth-1[[email protected] ~]# ip link set dev veth-1 up#为net-test中的veth-2配置IP并启动[[email protected] ~]# ip netns exec net-test ip addr add 10.0.0.2/24 dev veth-2[[email protected] ~]# ip netns exec net-test ip link set dev veth-2 up

When IP is configured for two NICs, a route is generated in the respective network namespace, which is viewed with IP route or route-n:

#在主机中查看路由[[email protected] ~]# route -nKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface...10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 veth-1...#在net-test中查看路由[[email protected] ~]# ip netns exec net-test route -nKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 veth-2

The above two routes indicate that the destination address 10.0.0.0/24 network IP packets are emitted from veth-1 and veth-2 respectively.

? Now Net-test this network namespace has its own network card, IP address, routing table and other information, it is equivalent to become a small "virtual machine". Test its connectivity to check that the configuration is correct.

? Ping Net-test's veth-2 network card from the host's veth-1 NIC:

? Ping the host's veth-1 Nic from the Net-test veth-2 NIC:

Connect the two network namespace.

? Many times, to build a complex network environment to test data, it is often trapped in the lack of sufficient resources to create virtual machines. After we have mastered the configuration of network namespace, we can solve this problem. Multiple isolated network namespace can be created in a simple way on a common machine, and then connected by virtual devices such as network cards, bridges, and so on, to form the desired topology.

? Let's demonstrate a simple example of connecting two network namespace to a Veth pair device. The process is as follows:

1, create two network namespace ns1, NS2, the name can be defined by itself:

[[email protected] ~]# ip netns add ns1[[email protected] ~]# ip netns add ns2[[email protected] ~]# ip netns lsns2ns1

2. Create Veth pair Device Veth-a,veth-b:

[[email protected] ~]# ip link add veth-a type veth peer name veth-b

3, put the network card into two network namespace, respectively:

[[email protected] ~]# ip link set veth-a netns ns1[[email protected] ~]# ip link set veth-b netns ns2

4. Start the two web sheets:

[[email protected] ~]# ip netns exec ns1 ip link set dev lo up[[email protected] ~]# ip netns exec ns1 ip link set dev veth-a up[[email protected] ~]# ip netns exec ns2 ip link set dev lo up[[email protected] ~]# ip netns exec ns2 ip link set dev veth-b up

5. Assigning IP:

[[email protected] ~]# ip netns exec ns1 ip addr add 10.0.0.1/24 dev veth-a[[email protected] ~]# ip netns exec ns2 ip addr add 10.0.0.2/24 dev veth-b

6. Verify Connectivity

? The two network namespace connected by the Veth pair device is like two machines connected directly through a network cable, and its topology diagram is as follows:

? Think about it, what if there are more network namespace need to connect? Is it necessary to introduce a virtual bridge, just like a docker network.

Configuring the Docker container network with IP commands

? In the previous article < "Understanding" to understand the Docker network core principles > introduced, Docker is the use of Linux namespace technology for resource isolation, the network is the same. When starting a Docker container with the default network mode (bridge mode), it must be a new Linux network namespace on the host. We can configure the network of the Docker container in accordance with the method of configuring the network in the namespace.

? First, start a Docker container named Test1:

[[email protected] ~]# docker run -itd --name test1 busybox

? Then use the IP Netns List command to see if you can see the new network namespace. After executing the command, it was found that the new network namespace was not seen. This does not mean that the Docker container does not create a network namespace, but that the IP netns command cannot be viewed, which is related to how the IP netns command works.

When you create two network namespace (ns1, NS2) using the IP netns command, you see/VAR/RUN/NETNS and ns1 in the NS2 directory:

[[email protected] ~]# ls -la /var/run/netns/total 0drwxr-xr-x  2 root root   80 Mar 19 18:25 .drwxr-xr-x 40 root root 1240 Mar 19 15:08 ..-r--r--r--  1 root root    0 Mar 19 18:22 ns1-r--r--r--  1 root root    0 Mar 19 18:22 ns2

The IP Netns List command looks for the network namespace under the/var/run/netns directory. Because the network namespace created by Docker does not create any options in this directory, some additional action is required to enable the IP command to manipulate the network namespace created by Docker.

? Each process under Linux belongs to a specific network namespace and looks at the difference between/pro/$PID/ns directories in different network namespace environments.

#/proc/self 链接到当前正在运行的进程[[email protected] ~]# ls -la /proc/self/ns/......lrwxrwxrwx 1 root root 0 Mar 19 19:17 net -> net:[4026531956]......#在ns1和ns2中[[email protected] ~]# ip netns exec ns1 ls -la /proc/self/ns......lrwxrwxrwx 1 root root 0 Mar 19 19:18 net -> net:[4026533018]......[[email protected] ~]# ip netns exec ns2 ls -la /proc/self/nslrwxrwxrwx 1 root root 0 Mar 19 19:18 net -> net:[4026533116]

? From the above can be found that different network namespace processes have different net:[] number. These numbers represent different network namespace, and processes with the same net:[] number belong to the same network NAMESAPCE. As long as you link the files created by Docker to the/var/run/netns directory, you can use the IP netns command to namesapce the following steps:

1. Use Docker inspect to view the PID of the Test1 container

[[email protected] ~]# docker inspect  --format ‘{{.State.Pid}}‘ test117037

2, if the/var/run/netns directory does not exist, it is necessary to manually create (usually have), and then create a soft link in the/var/run/netns directory, point to the Test1 container network namespace

[[email protected] ~]# ln -s /proc/17037/ns/net  /var/run/netns/test1

3. The test is successful

[[email protected] ~]# ip netns listtest1ns2ns1[[email protected] ~]# ip netns  exec test1 ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever226: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP     link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0    inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0       valid_lft forever preferred_lft forever

After you complete the above configuration, you can configure your Docker's network environment yourself. In addition to the IP netns command, there are tools to access the Linux network namespace, such as Nsenter. However, additional installation of this tool is required.

Play the pipework.

? Docker's existing network is relatively simple, scalable and flexible enough to meet many complex scenarios. There are many times when you need to customize the network of Docker containers. For example, in order to communicate between each node of the container, the communication between the nodes and the local host, it is simple to configure the Docker container network to a network segment of the local host network. Let's take a look at how it's achieved.

Configuring Docker containers in a local network environment

? If you want to have the Docker container and the container host on the same network, the containers and hosts should be in a two-tier network. is to connect the two machines to the same switch or to a different cascade switch. In the Virtual field shadow, the virtual Bridge can connect the container to a two-layer network, as long as the host's network card bridge to the virtual bridge, you can connect the container and host networks, and then assign a local LAN IP to the Docker container OK.

Let's go through an example to analyze this process: The local network is 172.18.18.0/24, the gateway is 172.18.18.1, the host IP is 172.18.18.34 (Nic ens160), to start a docker container named Test on this host, and configure it with IP 172.18.18.36. Since the network provided by Docker is not required, use the--net=none parameter to start the container. The operation is as follows:

1. Start a Test Container

[[email protected] ~]# docker run -itd --name test01 --network none busybox39ea5fac5ebb8bd25372d04efb6b662a18cd6fdf85105c22df0796087d776280

2. Create a bridge for the container connection Br0

[[email protected] ~]# brctl addbr br0[[email protected] ~]# ip link set dev br0

3, the host ens160 network Mezzanine Bridge to Br0, and the ens160 IP configuration on the br0. Because the author is a remote operation server, this step will cause the network to be disconnected, so here is placed in a command to execute

[[email protected] ~]# ip addr add 172.18.18.34/24 dev br0; > ip addr del 172.18.18.34/24 dev ens160; > brctl addif br0 ens160; > ip route del default; > ip route add default via 172.18.18.1 dev br0

4, find the test01 PID

[[email protected] ~]# docker inspect  --format ‘{{.State.Pid}}‘ test014557

5. Add the network namespace of the container to the/var/run/netns/directory

[[email protected] ~]# mkdir /var/run/netns[[email protected] netns]# ln -s /proc/4557/ns/net /var/run/netns/test01

6. Create a network card device for connecting the bridge and the Docker container

#将veth-a连接到br0网桥中[[email protected] ~]# ip link add veth-a type veth peer name veth-b[[email protected] ~]# brctl addif br0 veth-a[[email protected] ~]# ip link set dev veth-a up#将veth-b放在test的network namespace中，重命令eth0，并为其配置IP和默认路由[[email protected] ~]# ip netns exec test01 ip link set dev lo up[[email protected] ~]# ip link set veth-b netns test01[[email protected] ~]# ip netns exec test01 ip link set dev veth-b name eth0[[email protected] ~]# ip netns exec test01 ip link set eth0 up[[email protected] ~]# ip netns exec test01 ip addr add 172.18.18.36/24 dev eth0[[email protected] ~]# ip netns exec test01 ip route add default via 172.18.18.1

7, check the TEST01 network card situation, and test

[[email protected] ~]# ip netns exec test01 ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever9: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000    link/ether 66:fa:71:ba:0e:fb brd ff:ff:ff:ff:ff:ff link-netnsid 0    inet 172.18.18.36/24 scope global eth0       valid_lft forever preferred_lft forever[[email protected] ~]# ip netns exec test01 route -nKernel IP routing tableDestination     Gateway         Genmask         Flags Metric Ref    Use Iface0.0.0.0         172.18.18.1     0.0.0.0         UG    0      0        0 eth0172.18.18.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

To complete the configuration buckle, the network diagram for the Docker container and the host connection is as follows:

test01 containers are now accessible to the local host, and the TEST01 container can access the external network through the gateway 172.18.18.1 of the local network.

To parse the pipework.

As you can see from the above procedure, configuring the network of Docker containers is quite cumbersome. If you need to customize the Docker network frequently, you can write the above steps into a shell script so that it is easy to operate. In fact, there is now a tool that frees us from the tedious steps of Jerome Petazzoni, a Docker engineer, to release pipework on Githu. Pipwork claims to be a container SDN solution that connects containers in complex scenarios. In fact, with the continuous improvement of the Docker network, many of the functions of the Piipwork tool will be supported by Docker, so pipework was just one of the transition scenarios, as long as you know what to do. Here's a look at the features of pipework.

* supports the Linux bridge to connect to the container and configure the container IP
1. Download pipework

[[email protected] ~]# git clone https://github.com/jpetazzo/pipework

2. Place the pipework script in the specified directory,/usr/local/bin

[[email protected] ~]# cp ./pipework/pipework /usr/local/bin/

3. Configure the TEST01 container

[[email protected] /]# docker run -itd --name  test01 --network none busybox[[email protected] /]# pipework  br0 test01 172.18.18.36/[email protected]

The above configuration commands operate as follows:

• See if the host has a br0 bridge and it does not exist;
• Add a NIC named Eth1 to the test01 and configure the IP172.18.18.36/24;
• If there is already a default route in test01, delete it and set 172.18.18.1 as the default route L
• Connect the test01 container to the br0 on the previously created bridge;

This process is similar to the previous process of using IP command configuration, pipework is actually code written in Shell.

Pipework in fact, there are many other features, such as support open vswitch, support DHCP to obtain the container IP, and so on, as long as you understand its role and function, and other detailed functions are not introduced.

See here friends should have a better understanding of network namespace and pipework, like my article friends, please click on the top right corner of the "Attention" support!

Docker Advanced Networking Practices Linux Network namespace & pipework