Rsync+inotify implementation of real-time synchronization of files between servers in Linux system

Recently need to implement a dynamic backup of the files on the server, I do not want to do a manual backup every time, find a lot of information on the Internet, found that using rsync can be achieved, if you want to achieve real-time synchronization, you can also use the rsync+inotify combination, this article is done in combination.

Let's introduce Rsync and inotify.

1. rsync

Compared with the traditional methods of CP and Tar Backup, rsync has the advantages of high security, fast backup, support of incremental backup, and so on, rsync can solve the requirement of low-real-time data backup, such as regular backup file server data to remote server, regular data mirroring for local disk, etc.
With the expansion of application system scale, the security and reliability of data also put forward better requirements, rsync in the high-end business system also gradually exposed a lot of shortcomings, first, rsync synchronization data, need to scan all files after the comparison, for differential transmission. If the number of files reaches millions or even tens of thousands of levels, scanning all the files will be very time consuming. And what is changing is often a small part of it, which is a very inefficient way. Second, rsync cannot monitor and synchronize data in real time, although it can trigger synchronization through the way of the Linux daemon, but there must be a time difference between the two triggering actions, which results in inconsistent service and client data and the inability to fully recover data in the event of an application failure. Based on the above reasons, the rsync+inotify combination appeared!
2, INotify
Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the Inotify support, through Inotify can monitor the file system to add, delete, modify, move and other subtle events, using this kernel interface, Third-party software can monitor the file system under the various changes, and inotify-tools is such a third-party software.
In the above section, we mentioned that rsync can achieve triggered file synchronization, but through the crontab daemon mode trigger, synchronous data and actual data will be different, and inotify can monitor the file system changes, when the file has any changes, trigger rsync synchronization, This just solves the real-time problem of synchronizing data.
Specific people can refer to http://www.ibm.com/developerworks/cn/linux/l-ubuntu-inotify/index.html to learn.

Next we will start the installation, configuration, and testing of rsync and inotify.

The following is the structure of 2 servers, respectively, host name, IP, status, kernel, number of bits, synchronized directory, and 2 servers are CentOS 6.5 releases.

Host	Ip	Docoment
Server	192.168.1.21	/tmp
Client	192.168.1.22	/tmp

One, primary server (server)

Where the primary server needs to install Rsync and inotify, the primary server as the server, to the Backup server client transfer files

1. Install rsync

The version of the field rsync, if not, can be installed using Yum (you can also use the source code installation, here is not more introduction):

#安装rsync和xinetd and create the directory: Yum Install rsync xinetd# configuration xinetd: vi /etc/xinetd.d/= No start xinetd services: Service xinetd start

2. Establish Password Authentication file

The code is as follows:

Echo " rsync-pwd " >/etc/rsync. passwd #其中rsync-chmod /etc/rsync. passwd #无论是为了安全, or to avoid the following error, the password file needs to give 600 permissions

3, Installation INotify

[[Email protected]]# cd/usr/src/[email protected] src]#wgethttp//cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz[Email protected] src]#TarZXVF inotify-tools-3.14.Tar. gz [[email protected] src]# CD inotify-tools-3.14[email protected] inotify-tools-3.14]#./configure--prefix=/usr/local/inotify [[email protected] inotify-tools-3.14]# Make[email protected] inotify-tools-3.14]# Make Install

4. Create an rsync replication script

This feature is mainly the server side of the directory/TMP content, if modified (whether add, modify, delete files) can be monitored through the inotify, and through rsync real-time synchronization to the client's/TMP, the following is implemented through the shell script.

#!/bin/Bash host=192.168.1.22src=/tmp/des=Web user=WebUser/usr/local/inotify/bin/inotifywait-mrq--timefmt'%d/%m/%y%h:%m'--format'%T%w%f%e'-e modify,delete,create,attrib $src \| whileRead Files Do/USR/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/usr/local/rsync/rsync.passwd$src [email protected] $host:: $desEcho "${files} was rsynced">>/tmp/rsync.log2>&1  Done

Note: It is recommended that the rsync logs be placed in the other directory (not the backup directory).
Where host is the client's ip,src is the server side to real-time monitoring of the directory, DES is the name of the module authentication, need to be consistent with the client, user is to establish the password file authentication users.
Name this script rsync.sh, put it in the monitoring directory, for example, I put it under/tmp, and give 764 permissions

chmod 764 rsync. SH

Then run the script

sh /tmp/rsync. SH &

Keep in mind that only after Rsync is installed and started with rsync on the client side of the backup server, the rsync.sh script is started, or sometimes it appears full screen:

rsync:failed to connect to 192.168.1.22:connection refused (111)
Rsync error:error in Socket IO (code ten) at CLIENTSERVER.C (107) [sender=2.6.8]

We can also add the rsync.sh script to the boot entry.

Echo " /tmp/rsync.sh "

Second, backup server (Client)

1. Install rsync (the backup server only installs rsync)

#安装rsync和xinetd and create the directory: Yum Install rsync xinetd# configuration xinetd: vi /etc/xinetd.d/= No start xinetd services: Service xinetd start

2. Establish user and password authentication files

Echo " webuser:rsync-pwd " >/etc/rsync. passwd  chmod /etc/rsync. passwd #需要给密码文件600权限

3. Setting up rsync configuration file

Vim/etc/rsyncd.conf

UID =Root GID=Root usechroot=No max connections=TenStrict Modes=Yes PIDfile=/var/run/Rsyncd.pid Lockfile=/var/run/Rsync.lock Logfile=/var/log/rsyncd.log [web] path=/tmp/Comment= WebfileIgnore errors Read only=NoWriteonly =No hosts allow=192.168.10.220hosts Deny= *List=falseUID=Root GID=Root Auth users=WebUser Secretsfile=/usr/local/rsync/rsync.passwd 

The web is the server service side of the authentication module name, need to be consistent with the main server, the above configuration of my own server configuration, for reference.

4. Start rsync

Service xinetd Start
Chkconfig xinetd on   #设置开机自启动

Third, testing

Rsync and INotify are now installed on the server side, and Rsync is also installed on the client side of the backup server. Next you can test it:

Create a Test-rsync file in the server to see if the client can receive

Touch Test-rsync

Then see if the client side has the Test-rsync file, and the client side of the TMP directory file is the server side of the file is exactly the same. If consistent, it means that it has been set successfully.

Because a lot of information on the Web is reproduced, so it is not clear who the original author is, can only see the information I saw the link up, please forgive me!

Resources:

Http://www.jb51.net/article/57011.htm

Http://blog.sina.com.cn/s/blog_6e00431d0102visw.html

Rsync+inotify implementation of real-time synchronization of files between servers in Linux system