Rsync+inotify implementing real-time synchronization of directory files between servers (RPM)

Software Introduction:
1.rsync
Compared with the traditional methods of CP and Tar Backup, rsync has the advantages of high security, fast backup, support of incremental backup, and so on, rsync can
To address the requirements of low-real-time data backup, such as regular backup of file server data to the remote server, the local disk
Time to do data mirroring and so on.
As the scale of application system expands, and the security and reliability of data are better, rsync is in the high-end business system
Also gradually exposed a lot of shortcomings, first, rsync synchronization data, need to scan all files after the comparison, for differential transmission. Such as
If the number of files reaches millions or even tens of thousands of levels, scanning all files will be very time consuming. And what is changing is often one of the most
Less part of this is a very inefficient way. Second, rsync is not able to monitor and synchronize data in real time, although it can use Linux to keep
The process is triggered synchronously, but the two trigger action must have a time difference, which causes the server and client data to be
Inconsistent to fully recover data in the event of an application failure. Based on the above reasons, the rsync+inotify combination appeared!
2.inotify
Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, with the Linux kernel joining Inotify from 2.6.13
Support, through the Inotify can monitor the file system to add, delete, modify, move and other subtle events, the use of this kernel interface, the first
Three-party software can monitor the file system under the various changes, and inotify-tools is such a third-party software.
In the above section, we mentioned that rsync can implement triggered file synchronization, but it is triggered by the crontab daemon.
Synchronization of data and actual data will be different, and inotify can monitor various changes in the file system, when the file has any changes in the trigger
Rsync synchronization, which just solves the problem of real-time synchronization data.
3. Demand Analysis:
Due to the business needs of the company, it is necessary to realize real-time synchronization between two Web directories:

4. Realization mode rsync+inotify realize two file real-time synchronization
5. Primary server (server side, I am nginx here)
Where the primary server needs to install Rsync and inotify, the primary server as the server, to the Backup server client transfer files
6. Install Rsync
wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
Tar zxvf rsync-3.0.9.tar.gz
CD rsync-3.0.9
./configure--prefix=/usr/local/rsync
Make
Make install
7. After installation, establish the authentication password file:
Operation and Maintenance network original document Www.yunvn.com
echo "123456" >/usr/local/rsync/rsync.passwd
where "123456" is the password we set.
8. For security purposes we set the password file permissions to 600:
chmod 600/usr/local/rsync/rsync.passwd
9. Installing INotify
Wgethttp://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
Tar zxvf inotify-tools-3.14.tar.gz
CD inotify-tools-3.14
./configure--prefix=/usr/local/inotify
Make
Make install
10. Create an rsync replication script
The main function of the script is to put the server side of the directory/TMP content, if modified (whether adding, modifying, deleting files) can
enough to be monitored by inotify and synced to the client's/TMP in real time via rsync, which is implemented through shell scripting
#!/bin/bash
host=172.18.85.183
src=/tmp/
Des=apabi
User=apabicuser
/usr/local/inotify/bin/inotifywait-mrq--timefmt '%d/%m/%y%h:%m '--format '%t%w%f%e '-E
Modify,delete,create,attrib $SRC | While read files
Do
/USR/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/usr/local/rsync/rsync.passwd $src
[Email protected] $host:: $des
echo "${files} was rsynced" >>/var/log/rsync.log 2>&1
Done
If you put the Rsync.log to TMP (the backup directory) or send a copy of the problem, it is recommended that the rsync log put into
Other directories (not backup directories).
Where host is the client's ip,src is the server side to real-time monitoring of the directory, DES is a certified module name, need to be consistent with the client,
User is the Authenticated users in the Setup password file.
11. Name the script rsync.sh, put it in the monitoring directory, for example, I put it under/tmp, and give 764 permissions
chmod 764 rsync.sh
Remember to wait until the client installs and then run the script, no side will error
Sh/tmp/rsync.sh &
12. We can also add the rsync.sh script to the boot entry
[Email protected] tmp]# echo "/tmp/rsync.sh" >>/etc/rc.local
13. Install the Client configuration:
Installing rsync
Tar zxvf rsync-3.0.9.tar.gz
CD rsync-3.0.9
./configure--prefix=/usr/local/rsync
Make
Make install
14. Create a password file, and note that the information inside is consistent with the primary server
Operation and Maintenance network original document Www.yunvn.com
Cd/usr/local/rsync
echo "apabiuser:123456" >rsync.passwd
15. Set 600 permissions to the password file, or the startup service will error
chmod rsync.passwd
16. Create an rsync configuration file
Vim rsync.conf
UID = root
GID = root
Use chroot = no
Max connections = 10
Strict modes = yes
PID file =/var/run/rsyncd.pid
Lock file =/var/run/rsync.lock
Log file =/var/log/rsyncd.log
[Apabi]
Path =/tmp/
Comment = Apabi file
Ignore errors
Read Only = no
Write only = no
Hosts allow = 172.18.85.179
Hosts deny = *
List = False
UID = root
GID = root
Auth users = Apabiuser
Secrets file =/usr/local/rsync/rsync.passwd
Where Apabi is the name of the authentication module in the Server service, it needs to be consistent with the master server.
Name the configuration file rsync.conf, and put it in the/usr/local/rsync/directory.
16. Start the Rsync service

Files under Client tmp:

We create a test file on the server apabi20130117 see if we can automatically sync to the client

We'll go to the client now to see if there's automatic synchronization of the files we just created apabi20130117

Ok has synchronized past, below we test to delete a file on the server, whether the client can automatically delete
Delete file apabi20130117 on server:

Rsync+inotify implementing real-time synchronization of directory files between servers (RPM)