Summary of KVM virtualization Network Optimization Technology

The path from a complete data packet to a physical machine is:

Virtual Machine -- qemu virtual Nic -- virtualization layer -- kernel bridge -- physical Nic

650) This. width = 650; "Title =" clip_image002 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image002 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875258aFsR.gif "Height =" 100 "/>

KVM network optimization solution, in general, is to allow virtual machines to access fewer layers of the physical network card, until the physical network card is occupied separately, the same as the physical machine to use the physical network card, achieve the same network performance as physical machines.

Solution 1: Fully-virtualized nic and virtio

650) This. width = 650; "Title =" clip_image004 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image004 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875258doRQ.gif "Height =" 100 "/>

VirtioDifferences from full-virtualization NICs
Full-virtualization Nic is a fully simulated Nic on the virtualization layer. The semi-virtualization Nic is transformed to the operating system through the driver;
Viritio simply tells the virtual machine that you are running on a virtualization platform. Let's make some changes to achieve better performance on the virtualization platform;
650) This. width = 650; "Title =" clip_image006 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image006 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875259Qevw.jpg "Height =" 219 "/>

Use Cases of virtio
If you use viritio for a Windows virtual machine with a transient network disconnection, we recommend that you use a fully-virtualized Nic such as e1000 if the network pressure on the Windows virtual machine is low. If the network pressure is high, we recommend that you use technologies such as srivo or PCI device assignment. viritio is also evolving, and we hope that there will be fewer and fewer transient Disconnections for Windows.

KVM is designed for Linux systems by nature. For Linux systems, please feel free to use the viritio driver;

Solution 2 vhost_net macvtap Technology

650) This. width = 650; "Title =" clip_image008 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image008 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875260ItUl.gif "Height =" 100 "/>

Vhost_net allows Virtual Machine network communication to bypass the virtualization layer of user space and communicate with the kernel directly to provide virtual machine network performance;

Macvtap is a bridge that skips the kernel;

To use vhost_net, you must use a virtio semi-virtualization Nic;

Vhost_net Virtual Machine XML file configuration,

<interface type=‘bridge‘>     <mac address=‘‘/>     <source bridge=‘br0‘/>     <model type=‘virtio‘/><driver name="vhost"/>     <address type=‘pci‘ domain=‘0x0000‘ bus=‘0x00‘ slot=‘0x03‘ function=‘0x0‘/>   </interface>

If vhost_net is not used

<driver name="qemu"/>

Macvtap Virtual Machine xml configuration

<interface type=‘direct‘>      <mac address=‘00:16:3e:d5:d6:77‘/>      <source dev=‘lo‘ mode=‘bridge‘/>      <model type=‘e1000‘/>      <address type=‘pci‘ domain=‘0x0000‘ bus=‘0x00‘ slot=‘0x03‘ function=‘0x0‘/>    </interface>

Note: macvtap has poor performance on Windows virtual machines and is not recommended.

Comparison of vhost_net macvtap

The macvlan function is to configure multiple MAC addresses for the same physical network card. This allows you to configure multiple Ethernet ports in the software vendor.
Macvtap is used to replace the tun/TAP and bridge kernel modules. macvtap is based on the macvlan module and provides interfaces used by the tap devices in tun/TAP,
A vm using the macvtap Ethernet port can directly transmit data to the corresponding macvtap Ethernet port in the kernel through the tap device interface.
Vhost-net is an optimization for virtio. virtio was originally designed for frontend communication between the client system and vmm, reducing the number of non-root mode switches in the root mode in hardware virtualization mode.
Instead, after using vhost-net, you can further enter the root mode of the CPU, you need to enter the user State to send data to the TAP device and then switch back to the kernel state overhead, however, after entering the kernel state, you do not need to switch the kernel state user State, further reducing this kind of privileged switching, which layer does vhost-net belong, it is an optimization for L2 network data transmission.

Solution 3: Virtual Machine Nic exclusive

650) This. width = 650; "Title =" clip_image010 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image010 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875261ojlH.jpg "Height =" 554 "/>

How to configure the network adapter passthrough on a virtual machine

1Use lcpci devices to view PCI device information

04:00.0 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)04:00.1 Ethernet controller: Intel Corporation 82571EB Gigabit Ethernet Controller (rev 06)

You can also use virsh nodedev-list-tree to obtain information.

+- pci_0000_00_07_0|   ||   +- pci_0000_04_00_0|   |   ||   |   +- net_p1p1_00_1b_21_88_69_dc|   |  |   +- pci_0000_04_00_1|       ||       +- net_p1p2_00_1b_21_88_69_dd

2Run virsh nodedev-dumxml pci_1__04_00_0 to get the xml configuration information.

[[email protected]]# virsh nodedev-dumpxml pci_0000_04_00_0

<device><name>pci_0000_04_00_0</name><parent>pci_0000_00_07_0</parent><driver>   <name>e1000e</name></driver><capability type=‘pci‘>   <domain>0</domain>   <bus>4</bus>   <slot>0</slot>   <function>0</function>   <product id=‘0x105e‘>82571EB Gigabit Ethernet Controller</product>   <vendor id=‘0x8086‘>Intel Corporation</vendor></capability></device>

3. Edit the vm xml file and add PCI device information.

Domain bus slot function information is obtained from the XML file of dumpxml. Define the Virtual Machine and enable the Virtual Machine. Note that the attachment is a physical device and the corresponding driver must be installed in the system.


Programme 4 SR-IVO Technology


SriovPrinciple
SR-IVO is short for the single root I/O virtualization alization, is a standard for sharing PCIe to virtual machines, currently used on a large number of network devices, theoretically can also support other PCI devices, sriov requires hardware support.
650) This. width = 650; "Title =" clip_image012 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image012 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875261r6im.gif "Height =" 347 "/>
The following content is from the Oracle website. The link is
Http://docs.oracle.com/cd/E38902_01/html/E38873/glbzi.html
650) This. width = 650; "Title =" clip_image014 "style =" border-top: 0px; border-Right: 0px; Background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; padding-Right: 0px; "Border =" 0 "alt =" clip_image014 "src =" http://img1.51cto.com/attachment/201409/28/1724_1411875263R2Ez.gif "Height =" 369 "/>
Physical functions (PF)
Used for PCI functionality that supports SR-IOV functionality, as defined in SR-IOV specifications. PF contains the SR-IOV feature structure for managing SR-IOV functionality. PF is a full-featured PCIe feature that can be found, managed, and processed just like any other PCIe device. PF has full configuration resources and can be used to configure or control PCIe devices.
Virtual functions (VF)
A function associated with a physical function. VF is a lightweight PCIe function that can share one or more physical resources with physical functions and other VF associated with the same physical function. VF allows only configuration resources for its own behavior.
Each SR-IOV device can have a physical function (PF), and each pf can have up to 64,000 virtual functions (VF) associated with it ). PF can create VF through registers, which are designed to have specific properties for this purpose.
Once SR-IOV is enabled in PF, you can access the PCI configuration space of each VF through the PF bus, device, and function number (route ID. Each VF has a PCI memory space, which is used to map its register set. The VF device driver operates on the register set to enable its function and displays it as an existing PCI device. After creating VF, you can directly specify it to the IO guest domain or various applications (such as Oracle Solaris zones on bare metal platforms ). This feature allows virtual functions to share physical devices and execute I/O without the CPU and hypervisor software overhead.
Advantages of SR-IOV
SR-IOV standards allow efficient sharing of PCIe devices between IO guest domains. A SR-IOV device can have hundreds of virtual functions (VF) associated with a physical function (PF ). VF creation can be controlled dynamically by PF through registers designed to enable SR-IOV functionality. By default, the SR-IOV feature is disabled and PF acts as a traditional PCIe device.
Devices with SR-IOV capabilities can take advantage of the following:
Performance-access the hardware directly from the virtual machine environment.
Cost reduction-the saved capital and operating expenses include:
Energy Saving
Reduced the number of adapters
Simplified cabling
Reduces switch ports
SriovUse
Start the srivo kernel module
Modprobe igb
Activate virtual function VF
Modprobe igb max_vfs = 7
A Gigabit Nic can support up to 8 vf0-7, Gigabit Nic currently supports better is Intel i350, 82576s although also support sriov but only support virtual machine is Linux, Windows system does not support;
10G network card supports up to 64 vg0-63, Intel's new generation of 10G network card support sriov x520 x540;
If you need to reset VF, you can delete the module and reload it.
Modprobe-r igb
Permanently write configurations to the configuration file
Echo "Options igb max_vfs = 7">/etc/modprobe. d/igb. conf
The lspci command can be used to view multiple primary and sub-NICs.

# lspci | grep 82576    0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)    0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)    0b:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:10.7 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)    0b:11.5 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
 
The virtual machine can use the sub-nic exclusively by using the PCI Nic;

# virsh nodedev-list | grep 0b    pci_0000_0b_00_0    pci_0000_0b_00_1    pci_0000_0b_10_0    pci_0000_0b_10_1    pci_0000_0b_10_2    pci_0000_0b_10_3    pci_0000_0b_10_4    pci_0000_0b_10_5    pci_0000_0b_10_6    pci_0000_0b_11_7    pci_0000_0b_11_1    pci_0000_0b_11_2    pci_0000_0b_11_3    pci_0000_0b_11_4    pci_0000_0b_11_5
VM Nic XML file

<interface type=‘hostdev‘ managed=‘yes‘>        <source>          <address type=‘pci‘ domain=‘0‘ bus=‘11‘ slot=‘16‘ function=‘0‘/>        </source>      </interface>


Solution 5 Nic multi-queue
Centos 7Start to support multiple queues of virtio NICsTo greatly improve the Virtual Machine network performance. The configuration method is as follows:
XML Nic configuration for virtual machines
<interface type=‘network‘>     <source network=‘default‘/>        <model type=‘virtio‘/>         <driver name=‘vhost‘ queues=‘N‘/>    </interface>
N 1-8 supports up to 8 queues

Run the following command on the VM to enable multi-queue Nic
#ethtool -L eth0 combined M

M 1-n m less than or equal to n



I personally think that KVM network optimization solution focuses on hardware, and the solution of 10 Gigabit + sriov on hardware will become more and more popular, but the problem of online migration remains to be solved.
This article is from "xiaoli110 blog", please be sure to keep this source http://xiaoli110.blog.51cto.com/1724/1558984
Summary of KVM virtualization Network Optimization Technology