Rsync+inotify for real-time data synchronization

Purpose: in order to achieve real-time synchronization, we use INotify to monitor the changes in the directory that need to be synchronized, INotify is just a module of monitoring file changes in the kernel, provides the API for monitoring file changes, and to connect this API, The Inotify-tools tool needs to be installed. Can realize the new file, delete, modify, change attributes, etc., the function is very powerful. Now requires a server as a publisher to put the changed files through rsync in real time to sync to the content server.

First introduce the rsync and inotify, all in the online search information. First of all, the following rsync and inotify introduction is not my own writing.

1. rsync

2, INotify

Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the Inotify support, through Inotify can monitor the file system to add, delete, modify, move and other subtle events, using this kernel interface, Third-party software can monitor the file system under the various changes, and inotify-tools is such a third-party software. In the above section, we mentioned that rsync can achieve triggered file synchronization, but through the crontab daemon mode trigger, synchronous data and actual data will be different, and inotify can monitor the file system changes, when the file has any changes, trigger rsync synchronization, This just solves the real-time problem of synchronizing data.

3, test environment, two virtual host for testing function (Web use NGINX Deployment):

192.168.10.170 Deploy the content publisher on this host, install rsync-3.1.1.tar.gz and intoify-tools.tar.gz

192.168.10.190 deploy front-end servers on this host, install only rsync-3.1.1.tar.gz

Implementation: Automatically synchronizes data to the front-end server when content on the content publisher changes

Content Publishing Server:

4, installation rsync-3.1.1.tar.gz (192.168.10.170)

#tar XF rsync-3.1.1.tar.gz

#cd RSYNC-3.1.1/

#./configure--prefix=/usr/local/rsync/

#make && make Install

5. Create user files

#cd/usr/local/rsync/

#vim rsync.passwd

Hadoop # #只保存用户的密码

#chmod./RSYNC.PASSWD # #修改文件的权限

6, installation intoify-tools-3.1.14.tar.gz

#tar XF intoify-tools-3.1.14.tar.gz

#cd intoify-tools-3.1.4/

#./configure--prefix=/usr/local/intoify

#make && make Install

7. Create an rsync replication script

vim/tmp/rsync.sh

#!/bin/bash

host=192.168.10.190 # #需要数据同步的主机ip

SRC=/USR/LOCAL/NGINX/HTML/FTP # #本地需要同步的数据目录

Des=nginx

User=hadoop

/usr/local/inotify/bin/inotifywait-mrq--timefmt '%d/%m/%y%h:%m '--format '%T%w%f%e '-e modify,delete,create,attrib $ src | While read files

Do

/USR/LOCAL/RSYNC/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/usr/local/rsync/rsync.passwd $src [email Protected] $host:: $des

echo "${files} was rsynced" >>/tmp/rsync.log 2>&1

Done

8, change the script file execution permissions:

#chmod 764 rsync.sh

9. Let it run in the background

#sh/tmp/rsync.sh & or Nohup/tmp/rsync.sh &

Add The rsync.sh script to the boot entry

#echo "Nohup/tmp/rsync.sh &" >>/etc/rc.local

Front-end servers:

1, installation rsync-3.1.1.tar.gz (192.168.10.190)

#tar XF rsync-3.1.1.tar.gz

#cd RSYNC-3.1.1/

#./configure--prefix=/usr/local/rsync/

#make && make Install

2. Create user password file

#vim/usr/local/rsync/rsync.passwd

Hadoop:hadoop//Only password on server side, password and username on client side

3. change its permissions

#chmod rsync.passwd

4. Adding configuration Files

#vim/usr/local/rsync/rsync.conf

UID = root

GID = root

Use chroot = no

Max connections = 10

Strict modes = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsync.lock

Log file =/var/log/rsyncd.log

[Nginx]

Path =/tmp/# #数据同步到的目录

Comment = Web file

Ignore errors

Read Only = no

Write only = no

Hosts allow = 192.168.10.170

Hosts deny = *

List = False

UID = root

GID = root

Auth users = Hadoop

Secrets file =/usr/local/rsync/rsync.passwd

5. Start rsync

#/usr/local/rsync/bin/rsync--daemon--config=/usr/local/rsync/rsync.conf

6 . Add the Rsync startup script to the boot entry

# echo "/usr/local/rsync/bin/rsync--daemon--config=/usr/local/rsync/rsync.conf" >>/etc/rc.local

PS: Into the/etc/xinetd.d/directory to see if there are rsync files, there is an indication that rsync is managed by the XINETD process, you can start rsync by xinetd

#chkconfig rsync on

#service xinetd Restart

This article is from "Luo Chen's blog" blog, please be sure to keep this source http://luochen2015.blog.51cto.com/9772274/1696763

Rsync+inotify for real-time data synchronization