QEMU-IMG usage

RAW

Raw is the default format, simple format, easy to convert to other formats. File system support is required to support sparse file

Create image

# qemu-img Create-f Raw flat.img 10G
Formatting ' flat.img ', Fmt=raw size=10737418240

If we see LS

LS-LH flat.img
-rw-r--r--1 root root 10G June 22:27 flat.img

But it doesn't really occupy 10G.

# du-h Flat.img
0 flat.img

# qemu-img Info flat.img
Image:flat.img
File Format:raw
Virtual size:10g (10737418240 bytes)
Disk size:0

We can use DD to make sparse file.

The following command makes a non-sparse file

# dd If=/dev/zero of=flat1.img bs=1024k count=1000
1000+0 Records in
1000+0 Records out
1048576000 bytes (1.0 GB) copied, 0.66135 s, 1.6 GB/s

Block size is 1024k, total 1000 blocks

# qemu-img Info flat1.img
Image:flat1.img
File Format:raw
Virtual size:1.0g (1048576000 bytes)
Disk size:1.0g

The following command makes a sparse file

# dd If=/dev/zero of=flat2.img bs=1024k count=0 seek=2048
0+0 Records in
0+0 Records out
0 bytes (0 B) copied, 0.000141061 s, 0.0 kb/s

# qemu-img Info flat2.img
Image:flat2.img
File Format:raw
Virtual size:2.0g (2147483648 bytes)
Disk size:0

Seek means to set the end of a file in that place.

How do I copy a sparse file?

# dd If=/dev/zero of=flat3.img bs=1024k count=1 seek=2048
1+0 Records in
1+0 Records out
1048576 bytes (1.0 MB) copied, 0.0010721 s, 978 MB/s

# qemu-img Info flat3.img
Image:flat3.img
File Format:raw
Virtual size:2.0g (2148532224 bytes)
Disk size:1.0m

# CP--sparse=never Flat3.img flat3-never.img

So the copy takes up the entire 2G space.

# qemu-img Info flat3-never.img
Image:flat3-never.img
File Format:raw
Virtual size:2.0g (2148532224 bytes)
Disk size:2.0g

# CP--sparse=always Flat3.img flat3-always.img

So that the copy of the original 1M 0 will also be removed

# qemu-img Info flat3-always.img
Image:flat3-always.img
File Format:raw
Virtual size:2.0g (2148532224 bytes)
Disk size:0

# CP--sparse=always Flat3-never.img flat3-never-always.img

So the original 2G will be cleaned out.

# qemu-img Info flat3-never-always.img
Image:flat3-never-always.img
File Format:raw
Virtual size:2.0g (2148532224 bytes)
Disk size:0 Qcow2

Qcow2 is dynamic and has the following advantages over raw: Even though the file system does not support sparse files, the file size is small. Copy on write Snapshot compress encryption

The QCOW2 format is as follows:

2-level Loopup

The data for qcow2 is stored inside data clusters, and each cluster is a byte sector

In order to be able to manage these cluster,qcow2 saved two layers of table,l1 table to L2 table,l2 table to manage data cluster.

So the offset inside the image will be parsed into three parts, L1 table Pointer first find L1,L1 table pointer+ offset[0] is a L1 in entry, read out is L2 table Pointer, L2 table Pointer + offset[1] is a entry in L2, the data cluster Pointer is read out, and it is the location where the information is located. Reference Count

Each data cluster has refcount, because there may be multiple snapshot referencing each block copy-on-write

This is the use of backing file, a qcow2 image can save another disk image change without affecting the other image

root:/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7# qemu-img Info Disk
Image:disk
File Format:qcow2
Virtual size:20g (21474836480 bytes)
Disk size:450m
cluster_size:65536
Backing file:/var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610
Format Specific information:
compat:1.1
Lazy Refcounts:false

root:/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7# qemu-img info/var/lib/nova/instances/_base/ 45a5886083c924b1fbe412b2ad92f6ce47381610
Image:/var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610
File Format:raw
Virtual size:5.0g (5368709120 bytes)
Disk size:1.4g

The above output indicates that there is a raw image of/var/lib/nova/instances/_base/45a5886083c924b1fbe412b2ad92f6ce47381610, and we build a virtual machine based on him, But don't want to change the contents of this image, so we generated the image of Qcow2 based on it:/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk.

At first, the new image is empty, and the content read is read from the old image.

When a data cluster is written and changes occur, a new data cluster is created in the new image, which is the meaning of copy on write. Snapshot

Snapshot is actually an application of copy on write, but the two are subtly different.

Broadly speaking, there are two kinds of snapshot, one time internal snapshot, one is external snapshot

Internal snapshot is the snapshot maintained by snapshot table in Qcow2, and all snapshot are maintained in the same file.

In Monitor, SAVEVM, LOADVM are all operated for internal snapshot.

We create a virtual machine

# qemu-system-x86_64-enable-kvm-name ubuntutest-m 2048-hda ubuntutest.img-hdb ubuntutest1.img-boot menu=on-vnc:19 -net nic,model=virtio-net Tap,ifname=tap0,script=no,downscript=no

When we execute SAVEVM in monitor, ubuntutest.img and ubuntutest1.img will make a snapshot inside the image.

$ qemu-img Info ubuntutest.img
image:ubuntutest.img
File format:qcow2
Virtual size:5.0g (5368709120 byt ES)
Disk size:2.1g
cluster_size:65536
Snapshot list:
id        TAG                  VM size                 date        VM CLOCK
1         snapshot1               390M 2014-06-28 04:04:42   01:00:59.268
2         vm-20140630230902      239M 2014-06-30 23:09:02   00:04:07.783
Format Specific information:
    compat:1.1
     Lazy Refcounts:false

$ qemu-img Info ubuntutest1.img
Image:ubuntutest1.img
File Format:qcow2
Virtual size:4.0g (4294967296 bytes)
Disk size:134m
cluster_size:65536
Snapshot list:
ID TAG vm SIZE DATE VM CLOCK
2 vm-20140630230902 0 2014-06-30 23:09:02 00:04:07.783
Format Specific information:
compat:1.1
Lazy Refcounts:false

However, when the info block is in monitor, it still points to the two files.

The principle is that when a snapshot is hit, an entry is created in the snapshot table, but initially empty, containing a copy of the L1 table, and when L2 table or data cluster is changed, the original data is copied one copy, Maintained by snapshot's L1 table, and the original data cluster has changed, in situ.

External snapshot often use the above copy on write method, when playing snapshot, the current image will not change, create a new image, with the original image as the backing file, The virtual machine then uses the new image.

In Monitor, Snapshot_blkdev ide0-hd1 ubuntutest-snapshot.img qcow2, is used in this way, when this command is completed, in the monitor info block

And we found that there was one more file on the host machine.

$ qemu-img Info ubuntutest-snapshot.img
Image:ubuntutest-snapshot.img
File Format:qcow2
Virtual size:4.0g (4294967296 bytes)
Disk size:964k
cluster_size:65536
backing File:ubuntutest1.img
backing file Format:qcow2
Format Specific information:
compat:1.1
Lazy Refcounts:false

Backing file can be raw or qcow2, but once you hit snapshot, the new format is qcow2.

The two are very similar, slightly different is: for internal snapshot, just hit the snapshot, the original image collection is constant, the snapshot collection is empty, the next operation, written in the original image, will not be changed to join the snapshot collection For external snapshot, just after playing snapshot, the original image becomes the Snapshot,snapshot collection is complete, the new image is empty, the next operation, written in the new image, will change the addition of the new image collection. It uses ' qemu-img snapshot-c ' if the domain is offline and–disk-only were not specified it uses QEMU's ' SAVEVM ' if the Do Main is online and–disk-only were not specified it uses QEMU's ' Snapshot_blkdev ' if the domain is online and–disk-only is Specified

Virsh has command to support snapshot

For internal snapshot

root:/etc/neutron# Virsh snapshot-list instance-0000000a
Name Creation Time State
------------------------------------------------------------

root:/etc/neutron# Virsh snapshot-create instance-0000000a
Domain Snapshot 1404158715 created

root:/etc/neutron# Virsh snapshot-list instance-0000000a
Name Creation Time State
------------------------------------------------------------
1404158715 2014-07-01 04:05:15 +0800 Running

root# PS aux | grep instance-0000000a
libvirt+  9057  0.0  1.2 6846900 821876?      Sl   Jun26    5:34 qemu-system-x86_64-enable-kvm-name instance-0000000a-s-machine pc-i440fx-trusty,accel=kvm,usb=off-cpu sandybridge,+erms,+smep,+fsgsbase,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ Ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme-m 2048-realtime MLOCK=OFF-SMP 1,sockets=1,cores=1,threads =1-uuid 5d6cb926-5237-4b73-a7b4-e9af851023e7-smbios Type=1,manufacturer=openstack Foundation,product=OpenStack Nova,version=2014.1,serial=80590690-87d2-e311-b1b0-a0481cabdfb4,uuid=5d6cb926-5237-4b73-a7b4-e9af851023e7- No-user-config-nodefaults-chardev Socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-0000000a.monitor, Server,nowait-mon CHARDEV=CHARMONITOR,ID=MONITOR,MODE=CONTROL-RTC Base=utc,driftfix=slew-global Kvm-pit.lost_tick _policy=discard-no-hpet-no-shutdown-boot Strict=on-device piix3-usb-uhci,id=usb,bus=pci.0,Addr=0x1.0x2-drive file=/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk,if=none,id= Drive-virtio-disk0,format=qcow2,cache=none-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive= Drive-virtio-disk0,id=virtio-disk0,bootindex=1-netdev Tap,fd=31,id=hostnet0,vhost=on,vhostfd=33-device Virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:fe:2a:41,bus=pci.0,addr=0x3-chardev File,id=charserial0, Path=/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/console.log-device isa-serial,chardev= Charserial0,id=serial0-chardev Pty,id=charserial1-device Isa-serial,chardev=charserial1,id=serial1-device Usb-tablet,id=input0-vnc 0.0.0.0:8-K en-us-device Cirrus-vga,id=video0,bus=pci.0,addr=0x2-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
Root 17709 0.0 0.0 10468 924 pts/1 s+ 04:06 0:00 grep--color=auto instance-0000000a

root# qemu-img info/var/lib/nova/instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk
Image:/var/lib/nova/ Instances/5d6cb926-5237-4b73-a7b4-e9af851023e7/disk
File Format:qcow2
Virtual size:20g (21474836480 bytes)
Disk size:1.1g
cluster_size:65536
backing file:/var/lib/nova/instances/_base/ 45a5886083c924b1fbe412b2ad92f6ce47381610
Snapshot list:
id        TAG                  VM size                 date        VM CLOCK
1         1404158715              725M 2014-07-01 04:05:15  117:35:52.160
Format Specific information:
    compat:1.1
    lazy refcounts:false

For external snapshot

# Virsh Snapshot-create-as Instance01 snap1-instance01 "Instance01 snapshot description"--diskspec vda,file=/home/ Cliu8/images/instance01-snapshot.img--disk-only--atomic
Error:internal error:unable to execute QEMU command ' transaction ': Could not open '/home/cliu8/images/instance01.img ': C Ould not open '/home/cliu8/images/instance01.img ': Permission Denied:permission denied

From the online search, is because of the following questions:

-------------------------------------------------------------------------------------

https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1004606 Bug Description

When you attempt to create a Vsan snapshot onto a separate disk, this is fails with an error message. The error message is different depending on whether you use Snapshot-create or snapshot-create-as but both approaches fail .

The intention is-to-be able-take-a backup of a live VM by freezing the running image file and applying snapshot deltas To a separate file so the backup can save the filesystem image without it being modified during the backup. (i.e. practice in VM World)

Format being used is QCOW2 for the disk file which supports snapshots. Manually using Qemu-img to create a snapshot on the file does work without any problems.

If You use Snapshot-create-as:

Snapshot-create-as Winxppro3 snapname--disk-only--diskspec hda,snapshot=external,file=/srv/virtual-machines/ Tester/snap01.qcow2
Error:internal Error Unable to execute QEMU command ' blockdev-snapshot-sync ': A undefined error has ocurred

Or If you use the Snapshot-create and an XML file specification:

Virsh # snapshot-create Winxppro3 snapspec.xml
Error:argument Unsupported:un