kvm& The basic concept and development of Libvirt

Guide

Hello everyone, the theme of this Sholi sharing is kvm&libvirt basic concepts and development of the talk, the content of some messy loose, mainly based on their early collation of notes and practical sentiment, some content is inevitably biased, hope forgive me. First of all, we need to understand the basic knowledge, most of the content in the Sholi works have more detailed explanations, can read the reference.

KVM includes:

1. Kernel module Kvm.ko for the core virtual framework. 2. Contains processor-related modules KVM-INTEL.KO,KVM-AMD.KO3.KVM needs to use the modified customized QEMU software to provide user space tools * Kernel components are already included in the Linux kernel 2.6.20 * Some operating systems running in KVM still have some problems, you can view the operating system running compatibility status list provided by the KVM official website

Prerequisites for using KVM:

1.QEMU-KVM-RELEASE.TAR.GZ2.KVM-KMOD-RELEASE.TAR.BZ2, you need this stuff to compile your kernel Module 3. Intel processors that support VT technology or AMD processors that support SVM technology

Prerequisites for using QEMU:

1.zlib Library and header files 2.sdl library and header files 3.alsa library and header files, this is used to provide the virtual audio related features, the default is disabled, now do not know what state, you can use the--enable-alsa to enable 4.gnutls Library and header files, optional VNC TLS support, this feature is enabled by default, you can use--DISABLE-VNC-TLS to close the 5.kernel header file * Create, install, run the KVM virtual machine, need to use the user space provided by QEMU tools, previously was released separately, now the branch has been merged to become QEMU-KVM. *libvirt provides users with an abstract interface for interacting with different virtualization technologies.

libvirt Basic Concept:

1. Interacting with Linux virtualization Technology 2. Free, free 3. Stable C language API4. Other common Language bindings development interface (java,python,perl,php ... 5. For the DMTF virtual style 6.QMF proxy for AMQP, qpid information Systems

LIBIVRT Support degree:

Virtual technology: Kvm/qemu, XEN, LXC, OpenVZ, Usermodelinux, VB, ESX, HyperV, PowerVM, Parallels, Bhyre virtual network: Bridging, Nat,vepa,vnlink storage: IDE , Scsi,usb,fc,lvm,iscsi,nfs,fs

Libvirt support for a variety of development languages:

1. Direct support for C and C + +, other through the binding support 2.c#,java,ocaml,perl,php,python 3.libvirt Direct support Python, if it is installed through the installation package and not the source code compiled, you need to ensure that the appropriate package installed in Rhel, Called Libvirt-python, called Python-libvirt in Ubuntu, other platforms may have different names.

URI: A libvirt virtual environment for connecting or remotely connecting to a host, similar to a database-like connection string

1. Specify the URI to connect to Libvirt: By passing the name parameter to the Virconnectopen or Virconnectopenreadonly method: Virconnectptr conn = virconnectopenreadonly ("Xxx:///default"); 2. To save administrator time, you can configure the URI alias in the Libvirt client configuration file, the configuration file location/etc/libvirt/libvirt.conf (for root user) or $xdg_config_home/libvirt/ Libvirt.conf (for non-privileged users), use the following syntax to set the alias uri_aliases = ["Miaomiao=xxx://[email protected]/system", "Wangwang=xxx://[email Protected]/system "] aliases are made up of characters such as A-z,0-9,_, and URI aliases are applied to any application that opens Libvirt connection unless the parameter is Vir_connect_no_ Aliases is passed to Virconnectopenauth, and if the alias contains illegal characters, no alias lookups are made. If the URI passed to the Virconnectopen parameter is NULL, LIBVIRT will determine the URI with the following rule:   1. Environment variables: Libvirt_default_uri   2. Client Profile: Uri_default parameters  3. Probing each virtual Machine Manager knows to find an available

libvirt Basic Concepts:

Node: nodes, host, one is a separate physical machine for running virtual machine hypervisor: A software layer that can be used to virtualize a physical machine into multiple virtual machines with different configurations domain: virtual machine, in the case of container-level virtualization, is a subsystem, Run on a machine provided by hypervisor Libvirt: Provides a common software layer that securely and efficiently manages domain on node, while enabling remote management capabilities. * Use Libvirt Java bindings in Windows system, need to use Libvirt Java Library, Jna.jar library and corresponding platform DLL file, if error: Cannot find DLL or cannot find related module, need to install Virtviewer Added in Java code: System.setproperty ("Jna.library.path", "C:\\Program Files\\virtviewer\\bin"); And you need to place a copy of the Virt-0.dll file in the path above, named Virt.dll

SPICE:

The spice server is a VDI library implemented with Libspice, and VDI defines a series of interfaces for publishing virtual appliances. (e.g. display device, keyboard, mouse) and allow different spice components to interact with these devices, on the one hand, the server uses the spice protocol to communicate with the client, and on the other hand, the server interacts with the VDI host application, such as QEMU. The spice client is the interface that the end user faces.

QXL equipment and drives

When Libspice is used in conjunction with QEMU, the QEMU QXL PCI device can be used to enhance the remote display performance and enhance the client's graphics system. QXL devices require client QXL driver support. The spice protocol supports a communication channel that connects clients and server proxies, and when QEMU is used, the spice proxy is located on the client and the VDI port is a QEMUPCI device that can communicate with this agent.

Spice has six main passages:

Main channel: Control and Configuration display channel: Graphics command, graphics, video stream input channel: keyboard and mouse input cursor channel: Position device location with cursor shape playback channel: Service side sound plays recording channel on client: Audio capture from client

Spice image Compression features:

Spice itself provides a variety of image compression algorithms that can be selected at the time of server initialization or dynamically selected at run time. Quic is Spice's proprietary image compression algorithm, which is based on the sfalic algorithm, and the Lempel_zip (LZ) algorithm is also a spice-supported algorithm. Both the Quic and LZ are local algorithms that encode each image independently. Global LZ (GLZ) is another spice proprietary algorithm that is used in conjunction with the LZ's history-based Global dictionary table, GLZ can use repeating expressions between a large number of images to reduce traffic consumption and thus conserve bandwidth. Ideal for use in low-bandwidth WAN environments. Spice also provides an automatic selection of compression algorithm for each image of the working mode, through the image attribute heuristic selection Lz,glz,quic algorithm. In theory, LZ and Glz are more suitable for compressing synthetic images, and Quic is more suitable for compressing real images.

Video compression:

Spice uses a low-loss compression algorithm to compress images sent to the client, but the video stream is handled using a different approach. The spice server uses a self-inspired way to tag dynamic video regions and send them as video streams, encoded using MJPEG, which in some way saves traffic and improves spice performance, especially in WAN environments. In some cases, however, this heuristic can lead to low-quality image effects, such as the recognition of continuously updated text areas as video regions, resulting in poor image quality in some areas.

compiling from source code requires the following components:

*qpixman: A general-purpose library for controlling pixel areas, including low-level pixel control programs, also used by Cairo libraries, Cairo is a 2D graphics library for supporting multiple output devices, Qpixman is a lightweight modified version of Pixman * Qcairo:cairo is a vector graphics device-independent library, Qcairo is a lightweight modified version of Cairo *celt:celt is an audio compression algorithm for high-quality transmission of music, the signal will have a very small delay *ffmpeg: is a used for audio and video re-encoding, conversion, Stream the library. Includes LIBARCODEC Audio Video Coding library *log4cpp is a flexible logging library that can be targeted for files, syslog etc., followed by log4j after forming * about Windows virtual machine installation Virtio hard drive problem, One way is to install the system in the process of Virtio drive CD or floppy disk (WinXP), if the system has been installed, you can first create a small disk image, Qemu-img create-f qcow2 xxx.img 1G Edit Virtual machine configuration file, add this hard disk image, Set the parameter dev= ' VDC ' bus= ' virtio ' Start the system will find new hardware, select the corresponding Virtiosto driver in ISO image VIRTIOSCSI driver to shut down the VM, modify the virtual machine configuration, delete the small hard disk image, the original disk changed to dev= ' VDA ' bus= ' Virtio ' and delete the address section to start the System view driver. * The Libvirt-sock does not exist when you compile libvirt when you have an existing libvirt, the location is/run/user/1000/libvirt/libvirt-sock or/usr/local/var/run/ Libvirt/libvirt-sock, compiled and added to the LIBVIRTD daemon, use netstat to see if libvirt-sock is listening at the configuration file location.

Note When using Java's Libvirt development API:

* When writing a libvirt application using a Java program, when the virtual machine is in the pmsuspended state, domaininfo information cannot be obtained, and an array out-of-bounds error message appears. * When adding a device using the Attachdevice method, be aware that when flag=0, the execution results affect the current state, the current state is running, and only the runtime exists, and the current state is off to affect the shutdown profile. Flag=1, indicates that the runtime state is affected, and when this method is executed, the virtual machine must be in the active state. flag=2, the impact of the virtual machine persistence profile, but in the QEMU environment flag=1 will not support, the program will burst does not support hot-swappable, you can use domainupdatedeviceflags for the swap operation. After the persistent profile is updated, a shutdown restart is required for the new configuration to take effect. The Domain.getxmldesc () method gets the runtime configuration of the virtual machine, not the persistent configuration, so you will see that the deleted device is still there. * When updating a VM configuration using the Domainupdatedeviceflags () method, using UpdateFlags 0 or 1 will update the VM runtime configuration in the active state, which will be maintained after the virtual machine restarts, because rebooting the VM does not reload the persisted configuration. Reboot after shutdown and the configuration will revert to the original configuration. When UpdateFlags is used for 2 o'clock, there is a situation that cannot be updated, although the method succeeds, but the persisted profile cannot be updated. In the shutdown state, when using Updateflags=1, an error occurs because the update live status can only be performed in domain's active state. * Note that Libvirt does not support hot swapping of CDROM, Floppydisk drives, and when using the Attachdevice () method, when a flags of 0 or 1 o'clock is used, some device types such as CDROM for runtime modifications may return failures. The reason is that the hypervisor underlying driver is not supported. If the detach () method operation is performed on a device that is in block replication, hypervisor may block the operation of Detach (), in which case the copy operation needs to be stopped first using the Domainblockjobabort () method. * Depending on the hypervisor and device type, the removal of a device in a domain in active state may be performed asynchronously, that is, when you execute the detach () method, simply request the removal of a device, The actual time taken to remove this device may be the following time, which is done in conjunction with the client OS based on the virtual layer. This is often easy to ignore because it is sometimes seen in the configuration fileThis device has been removed, but hypervisor may not have actually removed this device, which may cause some subsequent operations to fail. To check if the device was actually successfully deleted, either re-use the Domaingetxmldesc () method or add an event handler for domain_event_id_device_removed if Detachdeviceflags () When the method returns, the device has been deleted and the event is triggered before the API call ends. To help existing clients work better in most cases, the API will attempt to turn the asynchronous delete operation after a period of time after the request into a synchronous delete operation, in other words, the API will wait for a while for the delete operation to complete in the case of an asynchronous operation. Note that hot-swappable devices are not maintained once domain enters the S4 state, which is the hibernation state, unless the persistent configuration file for domain is also modified. * When setting virtual machine maximum memory using the Setmaxmemory () method, Affect_current is used by default, that is, when domain is closed, the persistence profile is modified, and when domain is in the active state, Modifying the runtime configuration or runtime persistence configuration changes, which depends on the behavior of the hypervisor. For example, when domain is in active state in Libvirt, it will error when modifying the maximum memory, you must turn off domain and then modify the persistence configuration.

when using the API to develop the VM snapshot feature, there are several points to note:

If a virtual machine creates a snapshot with an internal snapshot or a system checkpoint, all hard disks of that virtual machine must create an internal snapshot, and QEMU does not support the creation of a hybrid snapshot, that is, a portion of the hard disk creates an internal snapshot, a portion creates an external snapshot, and of course a read-only disk, such as the optical drive, does not create The system checkpoint defaults to all the read-write drives using internal snapshots (the disks label content is ignored in the shutdown state). If you specify flags as Disk-only in the Create snapshot method, a hard disk that does not explicitly specify the snapshot mode uses an external snapshot by default, and QEMU does not currently support internal and external hybrid snapshots. When using a system checkpoint snapshot, there are requirements for the CPU feature set, and when the CPU feature set in the domain configuration has the INVTSC feature, the System Checkpoint snap error is executed at the domain runtime, the workaround is to remove the INVTSC feature or change the CPU model to custom. When a system-level snapshot is executed (on-state), if the profile label is snapshot=internal, the hard disk snapshot will also be executed by default using the internal method, whereas if the label is external, the drive will be executed by default using external mode. This is because the QEMU snapshot mechanism does not currently support internal and external snapshot blending mode, and the Libvirt internal mechanism automatically adjusts the disk snapshot mode to the same way as memory. Use external snapshots to be aware that the execution of an external snapshot file is a mirror from the original hard disk snapshot (base disk), after performing an external snapshot, libvirt changes the hard disk image in the domain configuration file to the newly created external snapshot file, so that the file can be used to store all the differentiated content after the snapshot is executed. Therefore, Libvirt does not allow direct deletion of external snapshots. External snapshots form a differential storage chain with an external snapshot that was previously created, and an internal snapshot does not form a differential storage connection to an external snapshot. After the snapshot is created, the snapshot profile contains the entire contents of the virtual machine configuration file for the restore. External snapshot                               Internal snapshot win7--- ---------------------->win7_snap-------------------->WIN7_SNAP2 (included inside the Win7_snap file) VMs                         &nbsp           external                          vm|--------------------->win7_snap3 VMS Win7 is a backingstore,win7_snap2 snapshot of win7_snap that exists in the Win7_snap file, Win7_snap win7_snap3 Backingstore, and memory external snapshots can be overwritten repeatedly. * In the shutdown state, perform any snapshot, cannot contain memory state, need to set the memory snapshot to No, because there is no memory running state, the memory is empty. * When creating a system checkpoint, an internal snapshot is created by default in the shutdown state, and internal snapshots must be used to create an external snapshot error, but using the Disk-only mode snapshot allows the creation of external snapshots in the shutdown state as well as internal snapshots. The default file name that is claimed by the external snapshot is called the original hard disk image. Snapshot name * Using the Disk-only method to create a snapshot for a running virtual machine, you must use the external method, and the label snapshot must be no, using an internal snapshot error. Internal snapshots and system checkpoint snapshots require that all hard disks participate in the execution of snapshots, use disk-only and external, and perform snapshots to perform snapshot policies separately for different hard disks. * Snapshots of virtual machines in the active state are performed using a system checkpoint snapshot, whether the snapshot is generated internally or externally depending on the snapshot value of the memory label. * To create a snapshot using diskonly mode or shutdown, the label snapshot must be no because neither of these snapshots can save the memory state.

matching of various snapshot modes and virtual machine states:

1. Virtual machine State, Diskonly mode: Memory=no (must), disk can use internal or external snapshots, and external snapshots can be customized for each disk to create a policy, the default creation of external snapshots. 2. Virtual machine running state, Diskonly mode: Memory=no (must), disk snapshot can only be created using external mode, do not support the creation of internal snapshots. 3. System checkpoint, virtual machine off state: Memory=no (must), disk snapshot can only use internal snapshots, and is true for all hard disks, can not be customized for each disk snapshot creation policy. 4. System checkpoint, virtual machine running state: memory=internal or External,disk snapshot creation mode depending on how memory snapshots are created, snapshots can only be created for all hard disks, regardless of internal or external snapshots, and custom creation policies for each drive are not supported. Why the snapshot mechanism is so confusing, because Libvirt and QEMU hold different views on the snapshot mechanism, but that's better, it's a matter of opinion, we can choose to use according to circumstances. PS: Internal Snapshot Advantage: No separate claims for extra files, no storage connection (in fact, internal metadata, similar mechanism), reduced file management complexity, Libvirt good support for internal snapshots. PS: Internal snapshot disadvantage: Slow to create, less maintenance for qemu upstream, and you must use file formats such as Qcow2 that support the cascading snapshot feature. There is a noticeable sense of pause when creating a snapshot VM at run time. PS: External Snapshot advantages: Create faster, support various source file formats (subsequent snapshots need to use QCOW2, etc.), create the client at runtime almost no time to pause, upstream qemu in the external snapshot development is more active PS: External snapshot disadvantage: A lot of snapshots will form a large number of storage files and storage connection, Each level uses cow to read and write, severely affects performance, file management is complex, and libvirt support for external snapshots is less aggressive.

FAQ memory snapshots, many mutually exclusive features, how to conduct mutual exclusion between the characteristics of each phase? Does snapshot failure fall back? Can I take a snapshot of the memory reuse scenario? Large memory snapshot time specifications about how long?

A: Snapshot recommendations based on my listed features to choose, do not recommend the snapshot function too much too complex, more snapshots will affect performance, if you have to do external snapshots, because the file is stored in Backingstore mode, it is best to commit the unused snapshot, memory reuse can take a snapshot, Large-size memory to do the snapshot needs to customize the source to do optimization, in fact, KVM virtualization just give us a basic function, a lot of features to meet customer needs need to customize the QEMU and KVM source code development.

Does spice have the ability to detect and identify USB devices? Is it possible for a KVM virtual machine to detect a USB device plugged into the host's USB port?

A: Spice USB device identification refers to the device connected on the terminal to identify as the function of the device on the virtual machine, which is a sub-function of spice USB REDIRECT implementation, as for the host peripherals, this is the problem with QEMU configuration.

Free to provide the latest Linux technology tutorials Books, for open-source technology enthusiasts to do more and better: http://www.linuxprobe.com/

kvm& The basic concept and development of Libvirt