Deep understanding of the OpenStack Network Architecture (1)

Original address: Https://blogs.oracle.com/ronen/entry/diving_into_openstack_network_architecture

Reprinted from: http://blog.csdn.net/halcyonbaby/article/details/41524447

Objective

The OpenStack network is powerful and relatively complex. This series of articles describes OpenStack's configuration through the Oracle OpenStack Techpreview, with various scenarios and examples explaining the various network components of OpenStack. The purpose of this article is to provide a panoramic view of the OpenStack network architecture and to show how each module works together. This is very helpful for OpenStack beginners and people who want to understand the principles of OpenStack networks. First, let's start by explaining some of the basics and giving examples.

According to the latest Icehouse version of the user survey, the neutron based on the Open Vswitch plugin is widely used in both production and POC environments, so in this series of articles we mainly analyze the configuration of this OpenStack network. Of course, we know that the OpenStack network supports many configurations, although Neutron+open vswitch is the most common configuration, but we never say it is the best or the most efficient way. Neutron+open Vswitch is just one example and is a good entry point for anyone wishing to understand the OpenStack network. Even if you plan to use other types of network configurations such as using different neutron plugins or not using neutron at all, this article is a good start for you to understand the OpenStack network.

The configuration we use in the example is a configuration provided by Oracle OpenStack Tech Preview. It's very simple to install, and it's a good reference. In this configuration, we use ETH2 as the virtual machine's network on all servers, and all virtual machine traffic uses this NIC. Oracle OpenStack Tech Preview uses VLANs for L2 isolation to provide tenant and network isolation, showing how we configure and deploy:

The first article will be slightly longer, and we will focus on some of the basic concepts of the OpenStack network. We will discuss several components such as open vswitch, network namespaces, Linux Bridge, Veth pairs, and so on. Note that these components are not intended to be fully described here, just to understand the OpenStack network architecture. These components can be further understood through other resources on the network.

Open VSwitch (OVS)

Used in Oracle OpenStack Tech preview to connect virtual machines and physical ports (eth2 in the example above), as shown in the deployment diagram above. OVS contains bridages and Ports,ovs bridges unlike the Linux Bridge (created with the Brctl command). Let's take a look at the structure of the OvS and use the following command:

[Plain]View Plaincopy

2. # Ovs-vsctl Show
4. 7ec51567-ab42-49e8-906d-b854309c9edf
6. Bridge Br-int
8. Port Br-int
10. Interface Br-int
12. Type:internal
14. Port "Int-br-eth2"
16. Interface "Int-br-eth2"
18. Bridge "Br-eth2"
20. Port "Br-eth2"
22. Interface "Br-eth2"
24. Type:internal
26. Port "Eth2"
28. Interface "Eth2"
30. Port "Phy-br-eth2"
32. Interface "Phy-br-eth2"
34. Ovs_version: "1.11.0"

We see a standard deployment of OVS on compute node with two bridges, each with several associated ports. The above example is on a compute node that does not have any virtual machines. We can see eth2 connected to a bridge called Br-eth2, we also see two called "Int-br-eth2" and "Phy-br-eth2" port, in fact, is a veth pair, as a virtual cable connection two bridages. We'll discuss Veth Paris in the rear.

When we create a virtual machine, a port is created on the Br-int Bridge, and the port is eventually connected to the virtual machine (we'll discuss the connection behind it). Here is the OVS structure after starting a virtual machine:

[Plain]View Plaincopy

2. # Ovs-vsctl Show
4. efd98c87-dc62-422d-8f73-a68c2a14e73d
6. Bridge Br-int
8. Port "Int-br-eth2"
10. Interface "Int-br-eth2"
12. Port Br-int
14. Interface Br-int
16. Type:internal
18. Port "qvocb64ea96-9f"
20. Tag:1
22. Interface "qvocb64ea96-9f"
24. Bridge "Br-eth2"
26. Port "Phy-br-eth2"
28. Interface "Phy-br-eth2"
30. Port "Br-eth2"
32. Interface "Br-eth2"
34. Type:internal
36. Port "Eth2"
38. Interface "Eth2"
40. Ovs_version: "1.11.0"

The "Br-int" Bridge now has a new port "qvocb64ea96-9f" connected to the VM and is marked as vlan1. Each NIC of a virtual machine needs to create a port on the "Br-int" bridge.

Another useful command in OvS is dump-flows, the following example:

[Plain]View Plaincopy

1. # Ovs-ofctl Dump-flows Br-int
2. Nxst_flow reply (xid=0x4):
3. cookie=0x0, duration=735.544s, table=0, n_packets=70, n_bytes=9976,idle_age=17, priority=3,in_port=1,dl_vlan=1000 Actions=mod_vlan_vid:1,normal
4. cookie=0x0, duration=76679.786s, table=0, N_packets=0, n_bytes=0,idle_age=65534, hard_age=65534, priority=2,in_port= 1 Actions=drop
5. cookie=0x0, duration=76681.36s, table=0, n_packets=68, n_bytes=7950,idle_age=17, hard_age=65534, Priority=1 actions= NORMAL

As mentioned above, the port connected by the VM uses VLAN tag 1. The port on the Virtual Machine Network (ETH2) then uses tag1000. The OvS modifies the VLAN of all the package between the VM and the physical network port. In OpenStack, the OVS agent controls the flows in open vswitch, which the user does not need to do. If you want to learn more about how to control the flow in open vswitch, you can refer to the description of Ovs-ofctl in http://openvswitch.org.

Network Namespaces (NETNS)

Network namespace is a cool feature of Linux, it has many uses. is widely used in OpenStack networks. The network namespace is a separate container for network configuration isolation, and the network cannot be seen by other namespaces. The network namespace can be used to encapsulate special network functions or to complete a complex network setup while isolating the network services. In Oracle OpenStack Tech Preview We use the latest R3 Enterprise kernel, which provides full support for Netns.

We show how to use the NETNS command to control network namespaces using the following example. To define a new namespace:

[Plain]View Plaincopy

1. # IP netns Add my-ns
2. # IP Netns List
3. My-ns

We said namespace is an isolated container, and we can do all kinds of things in namspace, such as the Ifconfig command.

[Plain]View Plaincopy

1. # IP NETNS EXEC MY-NS IFCONFIG -A  
2. lo        link encap:local loopback  
3.           loopback  mtu:16436  metric:1  
4.           rx packets:0  errors:0 dropped:0 overruns:0 frame:0  
5.            TX packets:0 errors:0 dropped:0 overruns:0  carrier:0  
6. collisions:0 txqueuelen:0  
7.            RX bytes:0  (0.0 b)   tx bytes:0   (0.0 b)   

We can run any command in namespace, such as the Tcddump command, which is very useful for debug, we use Ping, ssh, iptables command. Connecting namespace and external: There are many ways to connect to namespace and namespace direct connections, and we mainly gather in the methods used in OpenStack. OpenStack uses a combination of OvS and network namespace. OvS defines the interface, and then we add these interfaces to the namespace.

[Plain]View Plaincopy

2. # IP netns exec my-ns ifconfig-a
4. Lo Link encap:local Loopback
6. LOOPBACK mtu:65536 Metric:1
8. RX packets:0 errors:0 dropped:0 overruns:0 frame:0
10. TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
12. collisions:0 txqueuelen:0
14. RX bytes:0  (0.0 b) TX bytes:0 (0.0 b)
18. My-port Link encap:ethernet HWaddr 22:04:45:e2:85:21
20. Broadcast mtu:1500 Metric:1
22. RX packets:0 errors:0 dropped:0 overruns:0 frame:0
24. TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
26. collisions:0 txqueuelen:0
28. RX bytes:0  (0.0 b) TX bytes:0 (0.0 b)

Now we can add more ports to OvS Bridge and connect to other namespace or other devices like the physical NIC. Neutron uses network namespace to implement network services such as DHCP, routing, gateway, firewall, load balance, and so on. We will discuss more details in the next article.

Linux Bridge and Veth pairs

Linux Bridge is used to connect OvS ports and virtual machines. Ports is responsible for connecting OvS Bridge and Linux Bridge or both with virtual machines. Linux bridage is primarily used for security group enhancements. Security groups are implemented through iptables, and iptables can only be used for Linux bridage rather than OvS bridage.

Veth is a great tool for debugging network problems that are used extensively in OpenStack networks. The Veth pair is a simple virtual network cable, so it usually appears in pairs. Usually one end of the Veth pair is connected to bridge, the other end is connected to another bridge or left in use as a network port.

In this example, we will create some veth pairs, connect them to bridge and test connectivity. This example is for a typical Linux server instead of an OpenStack node: Create a Veth pair, and note that we define the names on both sides:

[Plain]View Plaincopy

2. # IP link Add veth0 type Veth peer name Veth1
6. # ifconfig-a
10. .
14. .
18. Veth0 Link encap:ethernet HWaddr 5e:2c:e6:03:d0:17
22. Broadcast Multicast mtu:1500 metric:1
26. RX packets:0 errors:0 dropped:0 overruns:0 frame:0
30. TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
34. collisions:0 txqueuelen:1000
38. RX bytes:0  (0.0 b) TX bytes:0 (0.0 b)
42. Veth1 Link encap:ethernet HWaddr e6:b6:e2:6d:42:b8
46. Broadcast Multicast mtu:1500 metric:1
50. RX packets:0 errors:0 dropped:0 overruns:0 frame:0
54. TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
58. collisions:0 txqueuelen:1000
62. RX bytes:0  (0.0 b) TX bytes:0 (0.0 b)
66. .
70. .

To make the example more meaningful, we will create the following configuration:

[Plain]View Plaincopy

1. Veth0 = veth1 =>br-eth3 = Eth3 ======> eth2 on another Linux server

BR-EHT3: A basic Linux bridge that connects Veth1 and Eth3eth3: a physical network port with no IP set connected to a network eth2: a physical network port on a remote Linux server, Connected to private network and configured with IP (50.50.50.1) Once we have created this configuration, we will 50.50.50.1 this remote IP via Veth0 Ping, thus testing the network connectivity:

[Plain]View Plaincopy

2. # Brctl ADDBR Br-eth3
6. # brctl AddIf br-eth3 eth3
10. # brctl AddIf br-eth3 veth1
14. # Brctl Show
18. Bridge name Bridge ID STP enabled interfaces
22. Br-eth3 8000.00505682e7f6 No Eth3
26. Veth1
30. # ifconfig Veth0 50.50.50.50
34. # ping-i Veth0 50.50.50.51
38. PING 50.50.50.51 (50.50.50.51) from 50.50.50.50 veth0:56 (+) bytes of data.
42. Bytes from 50.50.50.51:icmp_seq=1 ttl=64 time=0.454 ms
46. Bytes from 50.50.50.51:icmp_seq=2 ttl=64 time=0.298 ms
50. # ethtool-s veth1nic statistics:peer_ifindex:12# IP Link: 12:VETH0:MTU Qdisc pfifo_fast State up Qlen 1000

If the naming is not as obvious as in the example, causing us to be unable to support both ends of the Veth device, we can use the Ethtool command query. The Ethtool command returns the index number and uses the IP link command to view the corresponding device:

[Plain]View Plaincopy

1. # ethtool-s Veth1
3. NIC Statistics:
5. Peer_ifindex:12
7. # IP Link
9. .
11. .
13. 12:VETH0:MTU Qdisc pfifo_fast State up Qlen 1000

Summarize

In this article, we quickly learned about the ovs/network Namespaces/linux Bridges/veth. These components are used extensively in the OpenStack network architecture, and understanding these components helps us understand different network scenarios. In the next article, we'll learn how to communicate between virtual machines/virtual machines and external networks.

Deep understanding of the OpenStack Network Architecture (1)