The development of virtualization and the principle of realization--graphic explanation

Directory

• Directory
• Objective
• Virtualization of
• VM Virtual Machine
• Classification of virtualization
  • • x86 the relationship between CPU architecture and virtualization
  • Fully virtualized full virtualization
  • Semi-virtualized Paravirtualization
  • Hardware-assisted virtualization HVM
  • Memory Virtualization
    • Mapping implementation of memory virtualization
  • Bus Virtualization

Objective

The most common virtualization software available on the market now is VMware Workstation (vmware), VirtualBox (Oracle), Hyper-V (Microsoft), KVM (Redhat), Xen, and so on, which are collectively referred to as VMM ( Virtual machine Monitor), using a different virtualization implementation. The implementation of these virtualization methods can be divided into full virtualization, semi-virtualized, hardware virtualization, etc., this article is mainly to understand the implementation of these virtualization principles.

Virtualization of

the birth and realization of virtualization :
1961 IBM709 Machine realized a time-sharing system, the CPU is divided into a number of very short (1/100sec) times, each time slices are performing different tasks. By polling these time slices, you can virtualize or disguise a CPU as multiple CPUs, and make each virtual CPU appear to be running at the same time, which is the prototype of the virtual machine. Later system360 machines support time-sharing systems.
1972 IBM formally named the CTSS of the system370 machine as a virtual machine.
1990 IBM introduced the system390 machine support logical partition, will be a CPU divided into several parts (up to 10 copies), and each CPU is independent, that is, a physical CPU can be logically divided into 10 CPUs.

Until IBM put the time-sharing system open source, the personal PC finally came to the beginning of virtualization, and then the development of the above-mentioned virtual machine software. So far still a part of the virtual machine software application to the time- sharing system as a basis for the realization of virtualization.

The purpose of virtualization is to use logic to represent resources and to get rid of constraints of physical constraints. Improve the utilization of physical resources.
The principle of virtualization : By adding a virtualization layer (VMM) to the OS, the virtualization layer can encapsulate, isolate, and abstract the underlying (Hostos) hardware resources (physical CPU, memory, disk, Nic, graphics, and so on) into another form of logical resource, which is then provided to the upper layer ( Guestos) use. So you can understand that VMM is actually a middleware for contacting Hostos and Guestos, of course, virtualization can abstract a resource into multiple parts, or it can abstract multiple resources into a single copy.

Virtual machines implemented through virtualization are generally referred to as Guestos (customers), while physical hosts as guestos carriers are called Hostos (hosts).

VM Virtual Machine

• Efficient (>80%), stand-alone computer system provided by VMM
• Own virtual hardware (CPU, memory, network device, storage device)
• For the upper software, the virtual machine is the real machine
• Virtual Machine Monitor

The OS that satisfies the above conditions is the virtual machine.

features of the VM
Homogeneity: The nature of the VM is the same as that of the physical machine, e.g. ISA (instruction set schema instruction set Architecture) of the same CPU
Efficient: Performance is close to the physical machine, and most instructions executed on the VM should have permissions and ability to execute directly on the hardware, with only a handful of sensitive instructions handled by VMM. Performance efficiency for VMS in the previous article
Resource-controllable: VMM is absolutely controllable for both physical and virtual machine resources
Redhat has tested the efficiency of some application services running on virtual machines. Part of the report is as follows:

IBM DB2	SAP	ORACLE	JAVA	LAMP
Vm=host * 90%	Vm=host * 90%	Vm=host * 90%	Vm=host * 94%	Vm=host * 138%

Note: The efficiency of lamp running on VMS can be improved by splitting Apache, Php/python, and MySQL 3 app services into 3 different VMS.

Classification of virtualization

In the early stage of the development of virtualization, the two main schools are full-virtualization and semi-virtualized , each of which has advantages and disadvantages. If they can be properly applied in different environments, they will be able to give full play to the characteristics of both to achieve higher yields. As the two schools of divergence and competition intensified, the hardware manufacturers led by Intel also joined in the wave of virtualization, opened the era of hardware virtualization (Great sailing ERA), from the full virtualization and semi-virtualized road has gradually become closer to the trend. Then to the second generation of memory virtualization , the emergence and emergence of the third generation of bus virtualization , the current virtualization market is no longer the simple selling virtualization software as the main means of profit, but the integration of virtualization technology in a larger, more complete virtualization platform solutions. These include Redhat's Rhev, VMware's vsphere, and so on.

x86 the relationship between CPU architecture and virtualization

Before you understand the various implementations of virtualization, take a look at the general x86 CPU architecture

Note : In order to ensure the security of code execution, multi-user independence, and protect the normal operation of the OS, the CPU implementation state concept is realized. This can limit the access between different programs, avoid one program to get another program's memory data, but also avoid the program error operation of physical hardware. The general CPU is divided into the user state and the kernel state , x86 is subdivided in order to ring3~0 four kinds of states.

Ring3 User mode: The code that runs on the user state needs to be checked by the CPU, which can only access the virtual address (limited access memory) of the page in the Memory page table entry that specifies the user-state access, and can only access I/O in TSS Permission bitmap Specifies the ports that can be accessed by the user state. You cannot even access peripherals and cannot preempt the CPU. All user programs (application) are running in the user state. -- when an application running in a user state needs to invoke a hardware device, the CPU will invoke the kernel mentality code through a special interface, after which the user-state application can operate on the hardware device. If a user-configured application calls a hardware device directly, it is caught by the host OS and triggers an exception.

RING0 nuclear mentality (Kernel mode): is the host OS Kernel mode of operation, the code running in the kernel can be unrestricted access to system memory, device drivers, network interface, graphics interface and other peripheral devices. Only the host OS has unrestricted access to data for peripheral hardware devices such as hard disks, keyboards, etc. (drivers are required).

Virtualization Implementation Diagram :

Roughly speaking, both Guestos and VMM belong to applications running on RING3, and Guestos operating the hardware device passes the instruction to vmm,vmm to the Hostos after the instruction is monitored and detected. The Hostos will simulate the command issued by the Guestos as the core State instruction, and finally be processed by the CPU.
Note : When the process described above is very brief, it becomes very complex when you set up a process with full virtualization and semi-virtualized implementations.

Fully virtualized full virtualization

Guestos can run directly on a fully virtualized VMM without any modifications to itself, and the fully virtualized Guestos has full physical machine characteristics. Both VMM will simulate all of the abstract resources it needs for guestos, including CPU, disk, memory, NIC, graphics, and so on. e.g. Guestos when using a network card, the VMM emulated virtual NIC driver is invoked to operate the physical NIC.

In combination with the above-mentioned virtualization implementation diagram : Guestos is a virtual machine, and when we use Guestos, it is unavoidable to use the virtual device driver and the core scheduler in the Guestos to operate the hardware device. The difference with Hostos is that Hostos is running in the CPU's core mentality, which means that Hostos can operate the hardware device directly. However, the Guestos is running in the CPU user state, so it cannot directly operate the hardware device. To solve this problem, VMM refers to two mechanisms-the privilege release and the fall into the simulation .
privilege Cancellation : translation , when Guestos needs to use a kernel-state instruction, VMM dynamically captures and invokes several commands running in a non-nuclear mindset to simulate the effect of the kernel mentality command, thereby releasing the privileges of the kernel mentality. After releasing the privileges of the nuclear mentality, most of the instructions in the Guestos can be executed normally. However, this still does not solve the problem perfectly. Because there is also a sensitive instruction in an OS's instruction set (possibly a kernel state or a user state). At this point you need to fall into the simulation implementation.
fall into the simulation : Hostos and Guestos all contain sensitive instructions (reboot, shutdown, etc.), imagine if the Guestos command in reboot, but will hostos restart, this will be very bad. The simulation mechanism of VMM is designed to solve this problem. e.g. in Guestos, the reboot instruction that needs to run in the kernel state is executed, and VMM first acquires, detects, and determines the reboot instruction as a sensitive instruction. At this point, VMM will fall into the simulation, simulating the sensitive instruction reboot into a non-sensitive, non-kernel-minded "reboot" instruction that operates only on Guestos, and gives it to the CPU for processing. Finally, the CPU performs the Guestos restart operation accurately.

Because full virtualization simulates non-kernel instructions into a kernel-state instruction and then is processed by the CPU, it is less efficient than semi-virtualized, but the benefit of full virtualization is that it does not modify the Guestos, so a fully virtualized VMM can install most of the OS ( For the time being, only Linux, open Soralis, BSD and several other OS open source kernel code). Typical full virtualization software is--vmware, Hyper-V, kvm-x86 (complex instruction set).

two ways to implement full virtualization :
1). Full virtualization based on binary translation
2). Full virtualization based on scanning and patching

Semi-virtualized Paravirtualization

Para-Virtualization is a virtualization that requires guestos assistance. Because the Guestos kernel running in semi-virtualized has been modified in a special format. Semi-virtualized VMM is relatively simpler in handling sensitive instructions and kernel-state instructions. Guestos, which runs on a semi-virtualized VMM, needs to modify the kernel, primarily by modifying kernel-state directives including sensitive instructions in the Guestos kernel instruction set. When Hostos receives a guestos kernel-state instruction or sensitive instruction that has not undergone a semi-virtualized VMM simulation and translation processing, Hostos can also accurately determine whether the directive is guestos. This will allow the university to avoid the above problems. Typical semi-virtualized software with--xen, KVM-POWERPC (simple instruction set)

Semi-virtualized in addition to modifying the kernel there is another way to do this-in each of the Guestos installed semi-virtualized software,e.g. VMTools, Rhevtools.

Note : When running Windows with KVM, be sure to install the semi-virtualized drive tools, otherwise it will not work. Now the mainstream semi-virtualized drive is a generic semi-virtual machine driver virtio that is developed jointly by IBM and Redhat.

Hardware-assisted virtualization HVM

2005 Intel proposed and developed a virtualization technology that is directly supported by the CPU. This virtualization technology introduces new CPU operating modes and new instruction sets that enable VMM and Guestos to run in different modes (Vmm=root mode; Guestos=non-root mode), Guestos run in controlled mode, some of the original sensitive instructions in the controlled mode will all fall into the VMM, by the VMM to implement the simulation, so that some non-kernel-sensitive instructions to solve the fall- simulation problem , And the context saving recovery by the mode switch is done by the hardware, which greatly improves the efficiency of the context switch when it is caught in the simulation. The introduction of this technology makes it easy for the x86 CPU to achieve full virtualization. It is used by almost all the major schools of disagreement, including Kvm-x86,vmware ESX Server 3,xen 3.0.

categories of HVM :
1). intel–> vt-x
2). amd–> amd-v

Memory Virtualization

The original Guestos used virtual memory, not the lack of virtual memory to physical memory translation, affecting the efficiency of the virtual machine. Later Intel EPT AMD RVi indicated support for memory virtualization.

Mapping implementation of memory virtualization *

A–> virtual Address (VA), which refers to the linear address space that Guestos provides to its applications.
B–> Physical Address (PA), the pseudo-physical address that the virtual machine sees by VMM abstract
C–> Machine Address (MA), Real machine physical address, i.e. the address signal appearing on the address bus
mapping relationships for Memory addresses::
Guestos:pa = f (VA) #GuestOS维护着一套页表, responsible for the VA to PA mapping
Vmm:ma = g (PA) #VMM维护着一套页表, responsible for PA-to-MA mapping
The mapping from the virtual address to the machine address is implemented by the conversion method. During the actual run, the user program accesses the VA1, guestos the page table conversion to get PA1, and the VMM intervenes and uses the VMM page table to convert PA1 to MA1.

Bus Virtualization

category :
1). intel–> vt/d

2). amd–> Iommu
Bus virtualization can be implemented to divide a network card into several Guestos, each virtual machine 1/n, high performance, close to the real machine.
From a software perspective, the IO device is a stack of status registers that control registers, interrupts, and interacts with them.
Primary Virtualization : Device Interface full simulation, front-end-back-end emulation (Xen)?
Direct Partitioning : divide physical devices directly into the guest OS without having to go through VMM. Intel vt-d

Memory virtualization and bus virtualization further shorten the performance of Guestos and Hostos.

The development of virtualization and the principle of realization--graphic explanation