Real-time backup of data through Rsync+inotify

In the previous blog, I talked about using rsync to achieve data mirroring and backup, but to achieve real-time data backup, independent of rsync can not be achieved, this article describes how to achieve real-time data backup.

First, the advantages and disadvantages of rsync

Compared with the traditional CP and tar backup methods, Rsync has the advantages of high security, fast backup, support incremental backup, etc., through rsync can solve the real-time requirements of the data backup, such as regular backup file server data to the remote server, the local disk regularly do data mirroring.

With the expansion of the application system, the security and reliability of the data also put forward better requirements, rsync in high-end business system also gradually exposed a lot of deficiencies, first, rsync synchronized data, need to scan all files after the comparison, to carry out the differential transmission. If the number of files reaches millions or even millions of levels, scanning all files will be time-consuming. And what is changing is often a small part of it, which is a very inefficient way. Second, rsync can not real-time monitoring, synchronization of data, although it could be triggered by the Linux daemon process synchronization, but the two trigger action must have a time difference, which leads to server and client data may be inconsistent, not in the application of the failure to fully recover data. Based on the above reasons, the rsync+inotify combination appears!

Second, the first knowledge inotify

Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the Inotify support, through the Inotify can monitor the file system to add, delete, modify, move and so on a variety of subtle events, using this kernel interface, Third-party software can monitor the file system under the various changes in files, and inotify-tools is such a third-party software.

In the above section, we mentioned that rsync can implement trigger-type file synchronization, but the crontab daemon way of triggering, synchronized data and actual data will be different, and inotify can monitor the various changes in the file system, when the file has any changes, will trigger rsync synchronization, This just solves the problem of real-time synchronization data.

Installation of inotify Tools Inotify-tools

Because the INotify feature requires support from the Linux kernel, Before installing Inotify-tools, verify that the Linux system kernel is up to 2.6 13 or more, if the Linux kernel is lower than 2.6.13, you need to recompile the kernel to join the inotify support, or you can use the following method to determine whether the kernel supports inotify:

[Root@localhost webdata]# Uname-r

2.6.18-164.11.1.el5pae

[Root@localhost webdata]# Ll/proc/sys/fs/inotify

Total 0

-rw-r--r--1 root 0 04-13 19:56 max_queued_events

-rw-r--r--1 root 0 04-13 19:56 max_user_instances

-rw-r--r--1 root 0 04-13 19:56 max_user_watches

If you have the above three output, the system already supports inotify by default, and then you can start installing Inotify-tools.

You can download the appropriate Inotify-tools version to http://inotify-tools.sourceforge.net/, and then start compiling the installation:

[Root@localhost ~]# tar zxvf inotify-tools-3.14.tar.gz

Root@localhost ~]# CD inotify-tools-3.14

[Root@localhost inotify-tools-3.14]#./configure

[Root@localhost inotify-tools-3.14]# make

[Root@localhost inotify-tools-3.14]# make install

[Root@localhost inotify-tools-3.14]# ll/usr/local/bin/inotifywa*

-rwxr-xr-x 1 root root 37264 04-14 13:42/usr/local/bin/inotifywait

-rwxr-xr-x 1 root root 35438 04-14 13:42/usr/local/bin/inotifywatch

After the Inotify-tools installation is complete, inotifywait and Inotifywatch Two instructions are generated, where inotifywait is used to wait for a specific event on a file or set of files that can monitor any file and directory settings. and the entire directory tree can be monitored recursively.

Inotifywatch is used to collect monitored file system statistics, including how much inferior information occurs for each inotify event.