Rsync + inotify Real-time Data Synchronization instance

Rsync + inotify Real-time Data Synchronization instance

I. Why does the combination of rsync + inotify appear ??

Rsync defects:

1. When rsync implements a large amount of data backup, It scans the entire data file. In fact, only a small part of the changes are made, resulting in excessive use of system resources.

2. rsync cannot monitor and synchronize data in real time. Although it can be triggered by the linux daemon, there will be a time difference between the two triggering actions, as a result, the data on the server and the client may be inconsistent and the data cannot be completely restored when the application fails.

Inotify advantages:

Inotify is a powerful fine-grained and asynchronous file system event monitoring mechanism. Linux kernel has been added to Inotify since 2.6.13. Inotify can be used to monitor the addition and deletion of file systems, with this kernel interface, third-party software can monitor various file changes in the file system. inotify-tools is such a third-party software. Rsync can implement triggered file synchronization, but it is triggered by the crontab daemon. the synchronized data is different from the actual data, while inotify can monitor various changes in the file system, when the file changes, rsync synchronization is triggered, which solves the problem of real-time data synchronization.

Monitor host files and directories using inotifywait

Using inotify + rsync for Linux File batch update

Inotify-tools + rsync real-time file synchronization installation and configuration

Complete rsync synchronization Configuration

2. Install inotify-tools and briefly explain the basic usage
1. inotify installation conditions: the kernel version must be later than 2.6.13.
2. Determine whether the current version supports inotify Installation
# Uname-r
2.6.32-431. el6.x86 _ 64 (check the kernel version)
Or
# Ll/proc/sys/fs/inotify (check if there are any of the following three files. If yes, inotify is supported)
Total 0
-Rw-r -- 1 root 0 Aug 17 12:03 max_queued_events
-Rw-r -- 1 root 0 Aug 17 :03 max_user_instances
-Rw-r -- 1 root 0 Aug 17 :03 max_user_watches
3. Install inotify
(1) Compile and install
Download source package: http://inotify-tools.sourceforge.net (can also be downloaded from other places)
# Tar xf inotify-tools-3.14.tar.gz
# Cd inotify-tools-3.14
#./Conifgure
# Make & make install
# Ll/usr/local/bin/inotifywa * (check whether these two files are generated)
-Rwxr-xr-x 1 root 44271 Aug 17/usr/local/bin/inotifywait
-Rwxr-xr-x 1 root 41361 Aug 17/usr/local/bin/inotifywatch
(2) yum Installation
# Yum install inotify-tools
(Not described here)
The generated inotifywait and inotifywatch are described below.
After inotify-tools is installed, the inotifywait and inotifywatch commands are generated. inotifywait is used to wait for a specific event in a file or file set. It can monitor any file and directory settings, the entire directory tree can be monitored recursively.
Inotifywatch is used to collect statistics on the monitored file system, including the number of times each inotify event occurs.
4. inotify Parameters
Inotify defines the following interface parameters that can be used to limit the size of the kernel memory consumed by inotify. Because these parameters are all memory parameters, you can adjust the size of these parameters in real time based on application requirements. The following is a brief introduction.
/Proc/sys/fs/inotify/max_queued_evnets
Indicates the maximum number of events that can be queued in the inotify instance when inotify_init is called. Events that exceed this value are discarded, but the IN_Q_OVERFLOW event is triggered.
/Proc/sys/fs/inotify/max_user_instances
The maximum number of inotify instatnces that can be created by each real user ID.
/Proc/sys/fs/inotify/max_user_watches
The maximum number of directories that each inotify instatnces can monitor. If the number of monitored files is large, you need to increase the size of this value as appropriate, for example:
Echo 30000000>/proc/sys/fs/inotify/max_user_watches
5. inotifywait Parameters
Inotifywait is a monitoring wait event that can be used with shell scripts. The following describes some common parameters:
-M, that is, -- monitor, indicates that the event listening state is always maintained.
-R, that is, -- recursive, indicates the recursive query directory.
-Q, that is, -- quiet, indicates that the monitoring event is printed.
-E, that is, -- event. You can use this parameter to specify the event to be monitored. Common events include modify, delete, create, and attrib.

For more details, please continue to read the highlights on the next page:

1 2 Next Page