Learning notes for the US Mission network KVM virtualization Open Class

Qiu Jian, an open platform for KVM optimization technology and Meituan

Optimization Based on existing KVM options, video address: http://www.osforce.cn/course/77/learn#lesson/80

CPU optimization:

1. context switch (optimized from user State to kernel state); ring0-> nernel mode, ring 3-> User mode;

Set: Enable intel VT-X in host BIOS

2. cache-Node Binding Optimization

Bind the qemu process to a specific CPU node or core --- avoid L2/l3cache miss

Node Binding vs Core Binding

Set: Task Set

?

Memory Optimization

Address Translation and physical space optimization

1. Addressing-EPT (slat) Hardware acceleration

2. Addressing-hugepage reduces the size of the page table, reduces the cache-Miss of the query cache (TLB), and accelerates VM memory address conversion. The default pagesize is 4 kb/hugepage size is 2 MB;

Transparent hugepage: The khugepaged process periodically scans the memory, and automatically merges the common 4 kb pages that can be merged continuously into 2 MB hugepage. social self-method: sysctl-W sys. kernel. mm. transparent_huge ....

3. space optimization space-KSM: Kernel same-page merging

The kernel process ksmd periodically scans the memory and merges pages with the same content to reduce the usage of physical memory.

I/O optimization:

1. Storage 2. Data Exchange excuse between networks to simplify the process

There is no need to modify the guest OS for full virtualization alization technology, so there is no special cooperation between the virtualization layer and the guest OS layer, which affects the efficiency;

Virtio: a semi-virtualized I/O device framework that standardizes the data exchange excuse between guest and host, simplifies the process, reduces memory copying, and improves the I/O efficiency of virtual machines.

1. Storage

Virtio-BLK: Virtual PCI disk device based on virtio framework;

/Dev/vdx

Virtio-SCSI: Virtual SCSI disk device based on virtio framework? :/Dev/sdx

Cache mode Adjustment

Native AIO: Kernel AIO 2% ~ 3% to optimize the performance. Modify the command line parameters;

Block device Io scheduler optimization (via sysctl-w sys. Block. SDB. queue. scheduler ...):

Network Optimization:

1. virtio-Net: a virtual ethernet device based on virtio framework

Vhost_net optimization: Kernel process vhost_xxxx

Other Optimization Options:

1. CPU: Scheduler

2. Memory: NUMA

3. Storage: PCI-passthrough

4. Network: SR-IOV, PCI-Kernel

5. Improve hardware metrics-end-level solution

Related Questions

1. Role of VT-X: Context switching acceleration, fast switching from user State to kernel State to improve its performance;

2. Io isolation and network isolation QoS: Io isolation is set through virtio-BLK in two ways.

Network isolation isolates Bandwidth Resources;

3. Can a cgroup set each KVM qemu process or thread? -- You can find the corresponding qemu PID, restrict the PID, or use libvirt To make it easier

4. more than 95% of Meituan's businesses are on openstack;

5. bandwidth adjustment based on openswitch;

6. What is the KVM recovery mechanism in openstack? --- Snapshot mode, but occupies a large space;

7. Use gigabit network bandwidth without a Distributed File System

8. Monitoring: the host is monitored through scripts.