Multi-threaded compression migration technology in QEMU-KVM

Multi-threaded compression migration technology in QEMU-KVM

Guide

The current migration technique is to reach the destination of the virtual machine by directly writing bare memory data to qemufile, in which case the amount of data sent is large, resulting in higher migration time (total time) and black downtime (downtime). The method described in this article compresses the client memory before it is sent, and after the destination receives the memory, the data is decompressed to restore the client's memory.

1. Technical Principles

With a migration with compression technology, the total amount of data transferred is reduced by 60%, the total migration time is reduced by 70%+, and the downtime time is reduced by more than 50%. On the one hand, the compression/decompression process consumes CPU cycles and increases the time to migrate; On the other hand, the decrease in the total amount of data transferred reduces the migration time. In order to be able to perform high-speed compression, this technique uses multithreading concurrency in a way that improves the compression of the current virtual machine, using zlib to complete the compression/decompression work.

In the case of the same CPU, zlib official, the decompression speed is 4 times times the compression speed. In other words, if the source and destination processors in the same situation, so that the number of compressed threads is the number of decompression threads 4 times times the amount of resource consumption can be minimized, in order to more adapt to network conditions, the virtual machine introduced the compression level-Compression levels. The Compression level can be used to control the compression rate and compression ratio. A high compression ratio consumes more time, level 0 means no compression, Class 1 represents the optimal compression rate, and level 9 represents the best compression ratio (maximum compression time). We can select any level from level 0 to level 9.

2. Application scenarios of multi-thread compression dynamic migration technology

The compression/decompression time will consume CPU cycles. Therefore, avoid using this feature if the entire system CPU is being pressed to a very full state. When the network bandwidth is limited and the CPU resources are sufficient, the use of multi-thread compression dynamic migration technology will bring better results. When the network is plentiful and CPU resources are sufficient, the use of this technology will also reduce the total migration time.

3. Multithreading compression Migration technology Enable method Source side:

1. Start the virtual machine

/home/liufeng/qemu-system-x86_64-machine Accel=kvm-hda./disk0.img-m 2048-vnc 192.168.2.106:0-monitor stdio

2. Enable multi-threaded compression dynamic migration technology for energy end

A.) migrate_set_capabilitycompress on//enable to compress B.) Migrate_set_parametercompress-threads 12//12 compression threads C.) Migrate_set_parametercompress-level 1//compression level is 1 levels

3. Start the migration

Migrate-d tcp:192.168.2.105:6666

Destination side:

1. Start the virtual machine

/home/liufeng/qemu-system-x86_64-machine accel=kvm-hda/home/kvm/vm/disk/disk0.img-m 2048-vnc 192.168.2.105:0- Monitor Stdio-incoming tcp:192.168.2.105:6666

2. Enable multi-threaded compression dynamic migration technology at the destination

A.) migrate_set_capabilitycompress onb.) Migrate_set_parametercompress-level 1c.) Migrate_set_parameterdecompress-threads 3//3 Compressed threads

3. Wait for the migration to complete

4. Effect Verification Operating Environment:

Cpu:intel (R) Xeon (r) CPU e5-2650 v3 @2.30ghz
Logic core:40
Socket:2
ram:128g
Nic:1000baset/full
Host os:centoslinux release 7.2.1511 (Core) 64-bit
Guest os:centos Linux release 7.2.1511 (Core) 64-bit

A. Situation one: Unlimited bandwidth, sufficient CPU

	Original dynamic migration	Dynamic migration of multi-thread compression technology Compression level: 1 Number of compressed threads: 12 Number of decompressed Threads: 3
Total migration Time (msec):	9536	4466
Downtime Time (msec):	34	22
Amount of data transferred (KB)	307783	140445

Effect: Total migration time reduced 50%;downtime time decreased by 35%

B. Situation two: Bandwidth is limited, CPU is sufficient

	Original dynamic migration	Dynamic migration of multi-thread compression technology Compression level: 1 Number of compressed threads: 12 Number of decompressed Threads: 3
Total migration Time (msec):	11720	5652
Downtime Time (msec):	169	21st
Amount of data transferred (KB)	311554	140189

Effect: Reduced total migration time by 800% reduction in 200%,downtime time

5. Code Implementation Analysis

The Virtual machine Implementation Code analysis is as follows (this analysis is based on: QEMU 2.5):

1. In the boot migration process, if you find that the multithreaded compression technology is enabled, create a compressed thread

2. After the migration begins, use the multithreaded compression technology

There is Migration_thread () to perform the migration work, in the iterator and complete phase, if the discovery enables the multithreading compression technology, then through the Compress_page_with_multi_thread () completes the data compression and the transmission

3. Complete data compression via Zlib's Compress2 () function, and send via Qemu-file

Finally, the compression thread is activated in Compress_page_with_multi_thread (), the compression of the data is done through the zlib's Compress2 () function, and sent via Qemu-file

6. Can optimize the point

1. Compression algorithm

A. Currently using the open source Zlib library to complete compression, there are other compression libraries that can be provided to accommodate more scenarios

B. Commercial compression library with better efficiency

C. Hardware-assisted compression via FPGA

2. Compression policy

A. The virtual machine migration algorithm adapts to all networks, tests the network (whether it satisfies the above formula), and then forms a feedback factor input into the migration algorithm, which determines the compression algorithm, compression level, or non-compression used by the feedback factor, to shorten the downtime in all network conditions.

Free to provide the latest Linux technology tutorials Books, for open-source technology enthusiasts to do more and better: http://www.linuxprobe.com/

Multi-threaded compression migration technology in QEMU-KVM