Multi-thread compression and Migration Technology in QEMU-KVM, qemu-kvm Multithreading

Multi-thread compression and Migration Technology in QEMU-KVM, qemu-kvm Multithreading
Multi-thread compression and Migration Technology in QEMU-KVM

Guide

Currently, the Migration technology is to directly write raw memory data to QEMUFILE to transfer virtual machines. In this case, a large amount of data is sent, this results in a higher migration time (total time) and a lower time (downtime ). This document describes how to compress the client memory before sending it. After the target receives the memory, decompress the data to restore the client memory.

1. Technical Principles

After the migration with compression technology is used, the total amount of data transmitted will be reduced by 60%, the total migration time will be reduced by more than 70%, and the downtime will be reduced by more than 50%. On the one hand, the compression/decompression process will consume the CPU cycle and increase the migration time; on the other hand, the reduction of the total amount of transmitted data will reduce the migration time. In order to perform high-speed compression, this technology uses the multi-thread concurrency Method to Improve the compression of the current virtual machine, the use of ZLIB to complete the compression/Decompression work.

In the case of the same CPU, ZLIB officially provided that the decompression speed is four times faster than the compression speed. That is to say, if the source and target processors of the migration are the same, the number of compression threads is four times the number of decompression threads, so that the minimum resource consumption can be achieved, to better adapt to the network conditions, Compression level is introduced in the virtual machine. Compression level can be used to control the Compression rate and Compression ratio. A high compression rate consumes more time. level 0 indicates no compression, and level 1 indicates the optimal compression rate, level 9 represents the best compression ratio (the maximum compression time ). We can choose any level from level 0 to level 9.

2. Application scenarios of multi-threaded compression and dynamic migration Technology

The compression/Decompression Time will consume the CPU cycle. Therefore, if the CPU of the entire system is very full, avoid using this feature. When the network bandwidth is limited and the CPU resources are sufficient, the use of multi-threaded compression dynamic migration technology will bring better results. When the network is sufficient and the CPU resources are sufficient, using this technology will also reduce the total migration time.

3. multi-thread compression and migration technology enabling Method Source end:

1. Start the VM

/home/liufeng/qemu-system-x86_64 -machine accel=kvm -hda ./disk0.img -m 2048 -vnc 192.168.2.106:0 -monitor stdio

2. Enable multi-thread compression dynamic migration technology on the energy end

A.) migrate_set_capabilitycompress on // enable compression B.) migrate_set_parametercompress-threads 12 // 12 compression threads c.) migrate_set_parametercompress-level 1 // compression level 1

3. Start migration

migrate -d tcp:192.168.2.105:6666

Destination:

1. Start the VM

/home/liufeng/qemu-system-x86_64 -machine accel=kvm -hda /home/kvm/vm/disk/disk0.img -m 2048 -vnc 192.168.2.105:0 -monitor stdio -incoming tcp:192.168.2.105:6666

2. Enable the dynamic migration technology of multi-thread compression on the target end

A.) migrate_set_capabilitycompress onb.) migrate_set_parametercompress-level 1c.) migrate_set_parameterdecompress-threads 3 // 3 compression threads

3. Wait until the migration is completed

4. effect verification Running environment:

CPU: Intel (R) Xeon (R) CPU E5-2650 v3 @ 2.30 GHz
Logic core: 40
Socket: 2
RAM: 128 GB
NIC: 1000 baseT/Full
Host OS: CentOSLinux release 7.2.1511 (Core) 64-bit
Guest OS: CentOS Linux release 7.2.1511 (Core) 64-bit

A. Case 1: unlimited bandwidth and sufficient CPU

	Original Dynamic migration	Dynamic migration of multithreading compression technology Compression level: 1 Number of compressed threads: 12 Number of decompression threads: 3
Total migration time (msec ):	9536	4466
Downtime time (msec ):	34	22
Transfer Data Volume (KB)	307783	140445

Effect: The total migration time is reduced by 50%, and the downtime is reduced by 35%.

B. Case 2: limited bandwidth and sufficient CPU

	Original Dynamic migration	Dynamic migration of multithreading compression technology Compression level: 1 Number of compressed threads: 12 Number of decompression threads: 3
Total migration time (msec ):	11720	5652
Downtime time (msec ):	169	21
Transfer Data Volume (KB)	311554	140189

Effect: The total migration time is reduced by 200%, And the downtime is reduced by 800%.

5. Code Implementation Analysis

The Virtual Machine implementation code is analyzed as follows (this analysis is based on: QEMU 2.5 ):

1. When the migration process is started, if multi-threaded compression technology is found, a compression thread is created.

2. multithreading compression technology is used after migration starts

Migration_thread () is used for migration. In the iterator and complete phases, if multi-thread compression technology is enabled, compress_page_with_multi_thread () is used to compress_page_with_multi_thread () for data compression and transmission.

3. compress2 () function of zlib is used to compress the data and send

In the end, the compression thread is activated in compress_page_with_multi_thread (), the data is compressed by the compress2 () function of zlib, and the data is sent through the QEMU-FILE.

6. Optimization points

1. Compression Algorithm

A. Currently, the open-source zlib library is used for compression, and other compression methods can be provided to adapt to more scenarios.

B. Commercial Compression libraries have better efficiency

C. Hardware-assisted compression through FPGA

2. Compression Policy

A. the Virtual Machine migration algorithm adapts to all networks, tests the network (whether the above formula is met), and then forms a feedback factor input to the migration algorithm, the migration algorithm depends on the feedback factor on the compression algorithm, compression level, or no compression at all, to shorten the downtime in all network conditions.