Linux rsync+inotify for real-time synchronization of files between servers

Let's introduce Rsync and inotify.

1. rsync

Compared with the traditional methods of CP and Tar Backup, rsync has the advantages of high security, fast backup, support of incremental backup, and so on, rsync can solve the requirement of low-real-time data backup, such as regular backup file server data to remote server, regular data mirroring for local disk, etc.
With the expansion of application system scale, the security and reliability of data also put forward better requirements, rsync in the high-end business system also gradually exposed a lot of shortcomings, first, rsync synchronization data, need to scan all files after the comparison, for differential transmission. If the number of files reaches millions or even tens of thousands of levels, scanning all the files will be very time consuming. And what is changing is often a small part of it, which is a very inefficient way. Second, rsync cannot monitor and synchronize data in real time, although it can trigger synchronization through the way of the Linux daemon, but there must be a time difference between the two triggering actions, which results in inconsistent service and client data and the inability to fully recover data in the event of an application failure. Based on the above reasons, the rsync+inotify combination appeared!

2, INotify
Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the Inotify support, through Inotify can monitor the file system to add, delete, modify, move and other subtle events, using this kernel interface, Third-party software can monitor the file system under the various changes, and inotify-tools is such a third-party software.
In the above section, we mentioned that rsync can achieve triggered file synchronization, but through the crontab daemon mode trigger, synchronous data and actual data will be different, and inotify can monitor the file system changes, when the file has any changes, trigger rsync synchronization, This just solves the real-time problem of synchronizing data.
Specific people can refer to http://www.ibm.com/developerworks/cn/linux/l-ubuntu-inotify/index.html to learn.

Next we will start the installation, configuration, and testing of rsync and inotify.

The following is the structure of 2 servers, respectively, hostname, IP, synchronized directory, and 2 servers are CentOS 6.5 release.

Host	Ip	Docoment
Server	192.168.10.58	/tmp
Client	192.168.10.57	/tmp

One, primary server (server)

Where the primary server needs to install Rsync and inotify, the primary server as the server, to the Backup server client transfer files

1. Install rsync

#安装rsync和xinetd, and create the directory:install rsync xinetd# configuration xinetd:vi/etc/xinetd.d/rsync#disable =  Yes modified to disable = No to start xinetd Services: Service xinetd start     

2. Establish Password Authentication file

The code is as follows:

"rsync-pwd" >/etc/rsync.  passwd #其中rsync-600/etc/rsync.  passwd #无论是为了安全, or to avoid the following error, password files need to give 600 permissions      

3, Installation INotify

[[Email protected]]# cd/usr/src/[Email protected] src]#wget http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz [[email protected] src]# tar zxvf inotify-tools-3.14.  Tar. gz [[email protected] src]# CD inotify-tools-3.14 [[email protected] inotify-tools-3.14]#. Configure--prefix=/usr/local/inotify [[email protected] inotify-tools-3.14]# make [email Protected] inotify-tools-3.14]# make install         

4. Create an rsync replication script

This feature is mainly the server side of the directory/TMP content, if modified (whether add, modify, delete files) can be monitored through the inotify, and through rsync real-time synchronization to the client's/TMP, the following is implemented through the shell script.

#!/bin/Bash host=192.168.1.22src=/tmp/des=Web user=Webuser/usr/local/inotify/bin/inotifywait-mrq--timefmt‘%d/%m/%y%h:%m "--format " %t%w%f%e "-e Modify, Delete,create,attrib $src \ | while read files do/usr/ BIN/RSYNC-VZRTOPG--delete--progress--password-file=/usr/local/rsync/rsync.passwd $SRC [email protected] $host:: $des echo " ${files} was Rsynced" >>/tmp/rsync.log 2>&1 " Span style= "COLOR: #0000ff" >done            

Note: It is recommended that the rsync logs be placed in the other directory (not the backup directory).
Where host is the client's ip,src is the server side to real-time monitoring of the directory, DES is the name of the module authentication, need to be consistent with the client, user is to establish the password file authentication users.
Name this script rsync.sh, put it in the monitoring directory, for example, I put it under/tmp, and give 764 permissions

764 rsync. SH

Then run the script

Sh/tmp/rsync. Sh &

Keep in mind that only after Rsync is installed and started with rsync on the client side of the backup server, the rsync.sh script is started, or sometimes it appears full screen:

rsync:failed to connect to 192.168.1.22:connection refused (111)
Rsync error:error in Socket IO (code ten) at CLIENTSERVER.C (107) [sender=2.6.8]

We can also add the rsync.sh script to the boot entry.

"/tmp/rsync.sh  

Second, backup server (Client)

1. Install rsync (the backup server only installs rsync)

#安装rsync和xinetd, and create the directory:install rsync xinetd# configuration xinetd:vi/etc/xinetd.d/rsync#disable =  Yes modified to disable = No to start xinetd Services: Service xinetd start     

2. Establish user and password authentication files

"webuser:rsync-pwd" >/etc/rsync.  passwd600/etc/rsync.  passwd #需要给密码文件600权限    

3. Setting up rsync configuration file

Vim/etc/rsyncd.conf

UID =Root gid =Root useChroot =No max connections =10Strict modes =Yes PIDFile =/var/run/Rsyncd.pid LockFile =/var/run/rsync.lock log file =/var/log/ rsyncd.log [web] path =/tmp/ comment = web file ignore errors Read only = no write only = no hosts allow = 192.168.< Span style= "COLOR: #800080" >10.220 hosts deny = * list = Span style= "COLOR: #0000ff" >false uid = root gid = root auth users = WebUser secrets  File =/usr/local/rsync/rsync. passwd             

The web is the server service side of the authentication module name, need to be consistent with the main server, the above configuration of my own server configuration, for reference.

4. Start rsync

Service xinetd Start
Chkconfig xinetd on   #设置开机自启动

Rsync--daemon

third, testing

Rsync and INotify are now installed on the server side, and Rsync is also installed on the client side of the backup server. Next you can test it:

Create a Test-rsync file in the server to see if the client can receive

Touch Test-rsync

Then see if the client side has the Test-rsync file, and the client side of the TMP directory file is the server side of the file is exactly the same. If consistent, it means that it has been set successfully.

Linux rsync+inotify for real-time synchronization of files between servers