How to use LVM snapshot (snapshot) backup

LogicalVolumeManager (LVM) provides the "snapshot" (snapshot) function for any LogicalVolume (LV) to obtain the state consistency backup of a partition. When backing up a file in a certain state, an application may be accessing a file or database. This means that the file is in a state during backup, after the Logical Volume Manager (LVM) is backed up, it provides the "snapshot" (snapshot) function for any LogicalVolume (LV) to obtain the state consistency backup of a partition.

During a backup in a certain state, an application may be accessing a file or database. This means that the file is in the same state during the backup, and after the backup, files are in another state, resulting in inconsistent Backup. in this state, restoring database data is almost impossible.

The solution to the status is to mount the partition as read-only, and then back up data by locking the table-level writelocks of the database or even stopping the database. All these methods have no intention of seriously affecting service availability. LVMsnapshot can be used to obtain consistent backup without affecting the server availability.

It should be noted that the snapshot method is only valid for LVM and not for non-LVM file systems.

There are multiple ways to implement snapshot (refer to the last connection in the article). here we will talk about the implementation of "copy onwrite" in snapshot in LVM.

When a snapshot is created, only metadata (meta-data) of the data in the original volume is copied ). There is no physical copy of data during creation, so snapshot is almost real-time. when a write operation is executed on the original volume, snapshot tracks changes to the original volume block, at this time, the data to be changed on the original volume is copied to the space reserved by snapshot before the change. Therefore, the implementation of this principle is called copy-on-write ).

Before writing data to a block, CoW moves the original data to the snapshot space to ensure that all data is consistent when snapshot is created. For snapshot read operations, if the read data block is not modified, the read operation will be directly redirected to the original volume. if you want to read the modified block, read the blocks copied to snapshot.

In this way, the general file I/0 process is changed, that is, a cow layer is added between the file system and the device driver, which is like the following:
File I/0-> filesystem-> CoW?> Block I/O

The figure below may be easier to understand the CoW principle:

When CoW is adopted, the snapshot size does not need to be as large as the original volume. The size only needs to be considered in two aspects: from shapshot creation to release, estimate the size of the block change and the frequency of data update. Once the snapshot space record is full of the original volume block transformation information, the snapshot is immediately released and cannot be used, resulting in the snapshot being invalid. Therefore, it is very important to finish what you need to do in the snapshot lifecycle. Of course, if your snapshot size is as big as or even as large as the original volume, its life will be "same as the life of the day.

Snapshot has many other functions besides backup:

1) virtualization

When LVM2 is used, snapshots may not be read-only. This means that after creating a snapshot, you can mount and read and write the snapshot like a regular block device.

Because popular virtualization systems (such as Xen, VMWare, Qemu, and KVM) can use block devices as guest images, you can create complete copies of these images and use them as needed, they are like virtual machines with low memory usage. The advantage of this is that the deployment is fast (the snapshot creation time is usually less than a few seconds) and space saving (guest shares most of the data of the original image ).

The procedure is as follows:

1. create a logical volume for the original image.
2. use this LV as the disk image to install the guest virtual machine.
3. pause the VM. The memory image can be a regular file, and all other snapshots are stored in it.
4. create a read/write snapshot for the original LV.
5. use the snapshot volume as the disk image to generate a new virtual machine. Modify the network/console settings if necessary.
6. log on to the created virtual machine and modify the network settings/host name.

After completing these steps, you can access the created virtual machine. If you need another virtual machine, repeat steps 4 to 6 (so you do not need to reinstall the virtual machine ). You can also use a script to automatically perform these steps.

After using the virtual machine, you can stop the virtual machine and destroy the snapshot.

2) data tracing

You need to be cautious when performing some operations on a production system. even if you have done many tests in a simulated environment, there is no problem, but it cannot be guaranteed to succeed in the production environment, at this time, we will create a snapshot for the system, so that once a new operation encounters a problem, we will immediately go back to the snapshot creation time point. of course, you can also think that this is an extended backup usage.

Finally, we will give some examples to deepen our understanding of snapshot.

A) create a 20 m snapshot and perform some operations to check the CoW action.

Here is an example to illustrate how to create and use snapshot. Suppose we create a 20 m snapshot, which means that you can only change 20 m of data in the snapshot lifecycle.

The following command creates/dev/vg/lvdata-sp for/dev/vg/lvdata

# Lvcreate-L20M-s-n lvdata-sp/dev/vg/lvdata
Logical volume "lvdata-sp" created
The lvdata size is 20 MB.

# Lvdisplay/dev/vg/lvdata-sp

-Logical volume-

LV Name/dev/vg/lvdata-sp
VG Name vg
Lvuuid Yl0fQU-Ve9T-lfmp-xJPq-Uwrd-RVVM-lDDVz0
LV Write Access read/write
LV snapshot status active destination for/dev/vg/lvdata
LV Status available
# Open 1
LV Size 200.00 MB
Current LE 50
COW-table size 20.00 MB
COW-table LE 5
Allocated to snapshot 0.27%
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253: 0

The above Allocated to snapshot 0.27% is of our concern, indicating that there is still 99.73% space unavailable.

We try to create a 10 m file in lvdata and check the parameter value.

# Mount/dev/vg/lvdata/media/lvdata/
# Dd if =/dev/hda of =/media/lvdata/10 M bs = 1 M count = 10

10 + 0 records in
10 + 0 records out
10485760 bytes (10 MB) copied, 0.272393 seconds, 38.5 MB/s

# Lvdisplay/dev/vg/lvdata-sp

-Logical volume-

LV Name/dev/vg/lvdata-sp
VG Name vg
Lvuuid Yl0fQU-Ve9T-lfmp-xJPq-Uwrd-RVVM-lDDVz0
LV Write Access read/write
LV snapshot status active destination for/dev/vg/lvdata
LV Status available
# Open 0
LV Size 200.00 MB
Current LE 50
COW-table size 20.00 MB
COW-table LE 5
Allocated to snapshot 51.02%
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253: 0

"Allocated to snapshot51.02 %", as expected. At this time, snapshot has less than 10 MB of space left. if I create another 12 MB file on lvdata, what will happen?

# Dd if =/dev/hda of =/media/lvdata/12 M bs = 1 M count = 12

12 + 0 records in
12 + 0 records out
12582912 bytes (13 MB) copied, 0.288311 seconds, 43.6 MB/s
Device-mapper: snapshots: Invalidating snapshot: Unable to allocateexception.

An error is reported during file creation, and snapshot is invalid. Let's take a look at the details of the snapshot volume.

# Lvdisplay/dev/vg/lvdata-sp
/Dev/vg/lvdata-sp: read failed after 0 of 4096 at 0: input/output error

-Logical volume-

LV Name/dev/vg/lvdata-sp
VG Name vg
Lvuuid Yl0fQU-Ve9T-lfmp-xJPq-Uwrd-RVVM-lDDVz0
LV Write Access read/write
LV snapshot status INACTIVE destination for/dev/vg/lvdata
LV Status available
# Open 0
LV Size 200.00 MB
Current LE 50
COW-table size 20.00 MB
COW-table LE 5
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0

An I/0 error has occurred for the entire snapshot volume, and the snapshot status is "INACTIVE ".

Can it be mounted?

# Mount/dev/vg/lvdata-sp/media/snapshot/
Mount: you must specify the filesystem type

# Dmesg

Buffer I/O error on device dm-0, logical block 0
Buffer I/O error on device dm-0, logical block 1
Buffer I/O error on device dm-0, logical block 2
Buffer I/O error on device dm-0, logical block 3
Buffer I/O error on device dm-0, logical block 4
Buffer I/O error on device dm-0, logical block 5
Buffer I/O error on device dm-0, logical block 6
Buffer I/O error on device dm-0, logical block 7
Buffer I/O error on device dm-0, logical block 8
Buffer I/O error on device dm-0, logical block 9
Hfs: unable to find HFS + superblock

According to the error message of dmesg, the super block information is also lost.

Try to activate lvdata-sp

# Lvchange-ay/dev/vg/lvdata-sp

/Dev/vg/lvdata-sp: read failed after 0 of 4096 at 0: input/output error

Well, this snapshot has been released, so the only thing to do is to delete it.

# Lvremove/dev/vg/lvdata-sp

/Dev/vg/lvdata-sp: read failed after 0 of 4096 at 0: input/output error
Do you really want to remove active logical volume "lvdata-sp "? [Y/n]: y
Logical volume "lvdata-sp" successfully removed

B) use snapshot to back up the MySQL database online (or other databases)

The process is to first perform a flush operation, lock the table, create a snapshot, unlock it, back up the data, and finally release the snapshot. In this way, MySQL will hardly interrupt its operation.

Flush tables with read lock;
! Lvcreate? Size 100 m? Snapshot? Name snap/dev/VolGroup01/LogVol00
Unlock tables;

Next we will do some backup work.

Mkdir/snap
Mount/dev/VolGroup01/snap
# This is where you back up whatever you need from/snap, e. g. rsync (1)
Umount/snap
Lvremove/dev/VolGroup01/snap

Rmdir/snap