In addition, rsync + inotify enables Real-time Data Backup

In addition, rsync + inotify enables Real-time Data Backup

Advantages and disadvantages of rsync

Compared with traditional cp and tar backup methods, rsync has the advantages of high security, fast backup, and support for Incremental backup. rsync can meet the demand for data backup with low real-time requirements, for example, regularly back up the data on the file server to the remote server, and regularly perform data mirroring on the local disk.

As the scale of application systems continues to expand, rsync also puts forward better requirements on data security and reliability. In the high-end business system, rsync gradually exposes many shortcomings. First, during rsync data synchronization, You need to scan all files for comparison and transfer the data in a different amount. If the number of files reaches millions or even tens of millions, scanning all files will be very time-consuming. And the changes are often a small part, which is very inefficient. Secondly, rsync cannot monitor and synchronize data in real time. Although it can trigger Synchronization Through the linux daemon, there will be a time difference between the two triggering actions, as a result, the data on the server and the client may be inconsistent and the data cannot be completely restored when the application fails. For the above reason, the combination of rsync and inotify has appeared!

Inotify Introduction

Inotify is a powerful fine-grained and asynchronous file system event monitoring mechanism. Linux kernel has been added to Inotify since 2.6.13. Inotify can be used to monitor the addition and deletion of file systems, with this kernel interface, third-party software can monitor various file changes in the file system. inotify-tools is such a third-party software.

As we mentioned earlier, rsync can implement triggered file synchronization, but trigger through the crontab daemon process. The synchronized data is different from the actual data, inotify can monitor various changes in the file system. When there are any changes to the file, rsync synchronization is triggered, which just solves the problem of real-time data synchronization.

Install inotify and inotify-tools

The inotify feature requires support from the Linux kernel. Before installing inotify-tools, check whether the Linux kernel has reached 2.6.13 or later. If the Linux kernel is earlier than 2.6.13, you need to re-compile the kernel to support inotify. You can also use the following method to determine whether the kernel supports inotify.

Checks whether the system supports inotify.

# Linux shell

Uname-

Linux localhost 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3 x86_64 GNU/Linux # kernel later than 2.6.13 is supported

Ll/proc/sys/fs/inotify

Total usage 0

-Rw-r -- 1 root 0 May 19 20:02 max_queued_events

-Rw-r -- 1 root 0 May 19 20:02 max_user_instances

-Rw-r -- 1 root 0 May 19 20:02 max_user_watches

If the preceding three outputs are available, inotify is supported by the system by default, and inotify-tools can be installed.

Install inotify-tools

Debian system

Apt-get install inotify-tools

CentOS System

Yum install inotify-tools

Source code

Go to https://github.com/rvoicilas/inotify-tools/download source code compilation and installation.

Ll/usr/bin/inotifywa *

After inotify-tools is installed, the inotifywait and inotifywatch commands are generated. inotifywait is used to wait for a specific event in a file or file set. It can monitor any file and directory settings, the entire directory tree can be monitored recursively.

Inotifywatch is used to collect statistics on the monitored file system, including the number of times each inotify event occurs.

Inotify Parameters

Inotify defines the following interface parameters that can be used to limit the size of the kernel memory consumed by inotify. Because these parameters are all memory parameters, you can adjust the size of these parameters in real time based on application requirements.

Cat/proc/sys/fs/inotify/max_queued_evnets

Indicates the maximum number of events that can be queued in the inotify instance when inotify_init is called. Events that exceed this value are discarded, but the IN_Q_OVERFLOW event is triggered.

Cat/proc/sys/fs/inotify/max_user_instances

The maximum number of inotify instatnces that can be created by each real user ID.

Cat/proc/sys/fs/inotify/max_user_watches

The maximum number of directories that each inotify instatnces can monitor. If the number of monitored files is large, you need to increase the size of this value as appropriate, for example:

Echo 30000000>/proc/sys/fs/inotify/max_user_watches

Inotifywait-related parameters

Inotifywait is a monitoring wait event that can be used with shell scripts. The following describes some common parameters:

•-M, that is,-monitor, indicates that the event listening status is always maintained.

•-R, that is,-recursive, indicates the recursive query directory.

•-Q, I .e.-quiet, indicates that the monitoring event is printed.

•-E, that is,-event. You can use this parameter to specify the event to be monitored. Common events include modify, delete, create, and attrib.

For more detailed parameters, see the manual.

Man inotifywait # view Manual

-------------------------------------- Split line --------------------------------------

Rsync + inotify implement real-time synchronization and backup of Git data

Rsync for file backup Synchronization

Rsync synchronizes two servers

Remote synchronization of Rsync in CentOS 6.5

Use Rsync in Ubuntu Linux for data backup and Synchronization

Linux uses the Rsync client to synchronize directories with the server for backup

Rsync details: click here
Rsync: click here

This article permanently updates the link address: