Using Rsync+inotify-tools to realize real-time synchronous backup of data in Linux

Description

Operating system: CentOS 5.X

Source server: 192.168.21.129

Target server: 192.168.21.127,192.168.21.128

Objective: To synchronize the/home/www.111cn.net directory on the source server to the/home/www.111cn.net of the target server in real time

Specific actions:

Part one: Operate on two target servers 192.168.21.127,192.168.21.128 respectively

One, in two on the target server installation rsync service side

1. Close SELinux

Vi/etc/selinux/config #编辑防火墙配置文件

#SELINUX =enforcing #注释掉

#SELINUXTYPE =targeted #注释掉

Selinux=disabled #增加

: wq! #保存, exit

Setenforce 0 #立即生效

2, open Firewall TCP 873 port (rsync default port)

Vi/etc/sysconfig/iptables #编辑防火墙配置文件

-A rh-firewall-1-input-m state--state new-m tcp-p TCP--dport 873-j ACCEPT

: wq! #保存, exit

/etc/init.d/iptables Restart #最后重启防火墙使配置生效

3, install Rsync Server Software

Yum Install rsync xinetd #安装

Vi/etc/xinetd.d/rsync #编辑配置文件, set power-on start rsync

Disable = no #修改为no

: wq! #保存退出

/etc/init.d/xinetd start #启动 (CentOS is xinetd to manage rsync services)

4. Create rsyncd.conf configuration file

Vi/etc/rsyncd.conf #创建配置文件, add the following code

Log file =/var/log/rsyncd.log #日志文件位置, which automatically generates this file when you start rsync, without having to create it in advance

Pidfile =/var/run/rsyncd.pid #pid文件的存放位置

Lock file =/var/run/rsync.lock #支持max connections parameter

Secrets file =/etc/rsync.pass #用户认证配置文件, which saves the user name and password, which is later created

MOTD file =/etc/rsyncd. MOTD #rsync启动时欢迎信息页面文件位置 (File content customization)

[Home_www.111cn.net] #自定义名称

Path =/home/www.111cn.net/#rsync服务端数据目录路径

Comment = Home_www.111cn.net #模块名称与 [Osyunwei] Custom name is the same

UID = root #设置rsync运行权限为root

GID = root #设置rsync运行权限为root

port=873 #默认端口

Use chroot = no #默认为true, modified to No to increase backup of soft connections to directory files

Read Only = no #设置rsync服务端文件为读写权限

List = no #不显示rsync服务端资源列表

Max connections = #最大连接数

Timeout = #设置超时时间

Auth users = Home_www.111cn.net_user #执行数据同步的用户名, you can set multiple, separated by commas in English state

Hosts allow = 192.168.21.129 #允许进行数据同步的客户端IP地址, you can set multiple, separated by commas in English state

Hosts deny = 192.168.21.254 #禁止数据同步的客户端IP地址, you can set multiple, separated by commas in English state

: wq! #保存, exit

5, the creation of user certification files

Vi/etc/rsync.pass #配置文件, add the following

home_www.111cn.net_user:123456 #格式, Username: password, you can set multiple, one user name per line: password

: wq! #保存, exit

6. Set file permissions

chmod 600/etc/rsyncd.conf #设置文件所有者读取, Write permissions

chmod 600/etc/rsync.pass #设置文件所有者读取, Write permissions

7. Start Rsync

/etc/init.d/xinetd Start #启动

Service xinetd Stop #停止

Service xinetd Restart #重新启动

Part Two: Operating on the source server 192.168.21.129

First, install the rsync client

1. Close SELinux

Vi/etc/selinux/config #编辑防火墙配置文件

#SELINUX =enforcing #注释掉

#SELINUXTYPE =targeted #注释掉

Selinux=disabled #增加

: wq! #保存, exit

Setenforce 0 #立即生效

2, open Firewall TCP 873 port (rsync default port, as the client's rsync can not open 873 ports)

Vi/etc/sysconfig/iptables #编辑防火墙配置文件

-A rh-firewall-1-input-m state--state new-m tcp-p TCP--dport 873-j ACCEPT

: wq! #保存, exit

/etc/init.d/iptables Restart #最后重启防火墙使配置生效

System Yun-wei Www.111cn.net warm reminder: qihang01 original content copyright, reproduced please indicate the source and the original link

3. Install rsync client side software

Whereis rsync #查看系统是否已安装rsync, the following prompts show that you have installed

Rsync:/usr/bin/rsync/usr/share/man/man1/rsync.1.gz

Yum install xinetd #只安装xinetd即可, CentOS to manage rsync services in xinetd

Yum Install rsync xinetd #如果默认没有rsync, run this command to install Rsync and xinetd

Vi/etc/xinetd.d/rsync #编辑配置文件, set power-on start rsync

Disable = no #修改为

/etc/init.d/xinetd start #启动 (CentOS is xinetd to manage rsync services)

4. Create the authentication password file

Vi/etc/passwd.txt #编辑文件, add the following

123456 #密码

: wq! #保存退出

chmod 600/etc/passwd.txt #设置文件权限, only set the file owner to have read, write permission

5, test the source server 192.168.21.129 to the two target server 192.168.21.127,192.168.21.128 data synchronization between

Mkdir/home/www.111cn.net/ceshi #在源服务器上创建测试文件夹, and then run the following 2-line command on the source server

RSYNC-AVH--port=873--progress--delete/home/www.111cn.net/home_www.111cn.net_user@192.168.21.127::home_ Www.111cn.net--password-file=/etc/passwd.txt

RSYNC-AVH--port=873--progress--delete/home/www.111cn.net/home_www.111cn.net_user@192.168.21.128::home_ Www.111cn.net--password-file=/etc/passwd.txt

After the operation is completed, the two target servers are 192.168.21.127,192.168.21.128 on each other, and the Ceshi folder under the/home/www.111cn.net directory indicates that the data has been synchronized successfully.

Second, install Inotify-tools tools, real-time trigger rsync to synchronize

1. See if the server kernel supports inotify

Ll/proc/sys/fs/inotify #列出文件目录, the following contents show that the server kernel supports inotify

-rw-r--r--1 Root 0 Mar 7 02:17 max_queued_events

-rw-r--r--1 Root 0 Mar 7 02:17 max_user_instances

-rw-r--r--1 Root 0 Mar 7 02:17 max_user_watches

Note: Linux support inotify kernel minimum 2.6.13, you can enter the command: UNAME-A view the kernel

The CentOS 5.X kernel is 2.6.18 and is supported by default INotify

2. Installation Inotify-tools

Yum install make GCC gcc-c++ #安装编译工具

Inotify-tools Download Address: http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

Upload inotify-tools-3.14.tar.gz to/usr/local/src directory

Cd/usr/local/src

Tar zxvf inotify-tools-3.14.tar.gz #解压

CD inotify-tools-3.14 #进入解压目录

./configure--prefix=/usr/local/inotify #配置

Make #编译

Make install #安装

3, set the system environment variable, add soft connection

echo "Path=/usr/local/inotify/bin: $PATH" >>/etc/profile.d/inotify.sh

Source/etc/profile.d/inotify.sh #使设置立即生效

echo "/usr/local/inotify/lib" >/etc/ld.so.conf.d/inotify.conf

Ln-s/usr/local/inotify/include/usr/include/inotify

4, modify inotify default parameter (inotify default kernel parameter value is too small)

View system default parameter values

sysctl-a | grep max_queued_events

The result: Fs.inotify.max_queued_events = 16384

sysctl-a | grep max_user_watches

The result: Fs.inotify.max_user_watches = 8192

sysctl-a | grep max_user_instances

The result: fs.inotify.max_user_instances = 128

To modify a parameter:

Sysctl-w fs.inotify.max_queued_events= "99999999"

Sysctl-w fs.inotify.max_user_watches= "99999999"

Sysctl-w fs.inotify.max_user_instances= "65535"

Parameter description:

Max_queued_events:

INotify Queue Maximum Length, if the value is too small, there will be "* Event queue Overflow * *" error, resulting in inaccurate monitoring files

Max_user_watches:

How many directories are included in the file to sync, available by: Find/home/www.111cn.net-type D | Wc-l statistics, the max_user_watches value must be guaranteed to be greater than the statistical result (here/home/www.111cn.net is the synchronized file directory)

Max_user_instances:

Create INotify instance maximum per user

System Yun-wei Www.111cn.net warm reminder: qihang01 original content copyright, reproduced please indicate the source and the original link

5, create the script, real-time trigger rsync to sync

Vi/usr/local/inotify/rsync.sh #编辑, add the following code

======================================

#!/bin/sh

srcdir=/home/www.111cn.net/

Dstdir=home_www.111cn.net

Rsyncuser=home_www.111cn.net_user

Rsyncpassdir=/etc/passwd.txt

dstip= "192.168.21.128 192.168.21.127"

For IP in $dstip

Todo

RSYNC-AVH--port=873--progress--delete $srcdir $rsyncuser @ $ip:: $dstdir--password-file= $rsyncpassdir

Done

/usr/local/inotify/bin/inotifywait-mrq--timefmt '%d/%m/%y%h:%m '--format '%T%w%f%e '-e close_write,modify,delete,  Create,attrib,move $srcdir | While read file

Todo

For IP in $dstip

Todo

RSYNC-AVH--port=873--progress--delete $srcdir $rsyncuser @ $ip:: $dstdir--password-file= $rsyncpassdir

echo "${file} was rsynced" >>/tmp/rsync.log 2>&1

Done

Done

======================================

chmod +x/usr/local/inotify/rsync.sh #添加脚本执行权限

Script parameter Description:

srcdir=/home/www.111cn.net/#源服务器同步目录

Dstdir=home_www.111cn.net #目标服务器rsync同步目录模块名称

Rsyncuser=home_www.111cn.net_user #目标服务器rsync同步用户名

Rsyncpassdir=/etc/passwd.txt #目标服务器rsync同步用户的密码在源服务器的存放路径

dstip= "192.168.21.127 192.168.21.128" #目标服务器ip, multiple IP separated by space

/tmp/rsync.log #脚本运行日志记录

6, set the script on the automatic execution of the boot

Vi/etc/rc.d/rc.local #编辑, adding a row at the end

Sh/usr/local/inotify/rsync.sh & # Set up to run the script in the background automatically

: wq! #保存退出

7, test inotify real-time trigger rsync synchronization script is normal operation

Create a file on the source server 192.168.21.129 Inotify_rsync_ceshi

Mkdir/home/www.111cn.net/inotify_rsync_ceshi

Reboot source server: 192.168.21.129

After the system starts, check to see if there are Inotify_rsync_ceshi folders under the/home/www.111cn.net of the two target servers 192.168.21.127,192.168.21.128

Then create the folder on the source server 192.168.21.129 inotify_rsync_ceshi_new

Mkdir/home/www.111cn.net/inotify_rsync_ceshi_new

Continue to see if there are inotify_rsync_ceshi_new folders under the/home/www.111cn.net of two target servers 192.168.21.127,192.168.21.128

If all the above tests pass, the inotify real-time triggering rsync synchronization script is working properly.

At this point, Linux Rsync+inotify-tools implementation of real-time data synchronization completed.

Extended reading:

============================================

INotify parameters

-M is to keep listening all the time

-R is a recursive view of the directory

-Q is printing out events

-e Create,move,delete,modify,attrib refers to the "Listen for Create move delete Write permission" event

rsync parameters

============================================

-V,--verbose verbose mode output

-Q,--quiet thin output mode

-C,--checksum Open the check switch to force the file transfer to verify

-A,--archive archive mode, which represents the transfer of files recursively and maintains all file attributes equal to-rlptgod

-R,--recursive the subdirectory in recursive mode

-R,--relative using relative path information

-B,--backup creates a backup, which means that the old file is renamed to ~filename for the same file name already exists for the purpose. You can use the--suffix option to specify different backup file prefixes.

--backup-dir store backup files (such as ~filename) in the directory.

-suffix=suffix defines the backup file prefix

-U,--update only updates, skipping all files that already exist in DST, and the file time is later than the file you want to back up. (Do not overwrite the updated file)

-L,--links retention soft link knot

-L,--copy-links to treat soft links like regular files

--copy-unsafe-links only copies links that point beyond the src Path directory tree

--safe-links ignores links to the SRC path tree

-H,--hard-links keep hard links

-P,--perms keep file permissions

-O,--owner keep file owner information

-G,--group keep file group information

-D,--devices maintain device file information

-T,--times keep file time information

-S,--sparse special handling of sparse files to save DST space

-N,--dry-run which files will be transmitted

-W,--whole-file copy files without incremental detection

-X,--one-file-system do not cross file system boundaries

-B, the block size used by the--block-size=size test algorithm is 700 bytes by default

-E,--rsh=command specifies the use of rsh, SSH mode for data synchronization

--RSYNC-PATH=PATH Specifies the path information for the rsync command on the remote server

-C,--cvs-exclude use the same method as CVs to automatically ignore files to exclude files that you do not want to transfer

--existing updates only those files that already exist in DST, not the newly created files

--delete Delete files that are not in the DST SRC

--delete-excluded also deletes the receiving end of files that are excluded by the option specified

--delete-after after transmission is over

--ignore-errors in time IO errors are also deleted

--max-delete=num Delete NUM files up to

--partial retains files that are not fully transmitted for some reason, to expedite subsequent transmissions

--force forcibly deletes a directory, even if it is not empty

--numeric-ids does not match the user and group IDs of numbers to user and group names

--timeout=time IP timeout time in seconds

-I,--ignore-times not skip those files that have the same time and length

--size-only when deciding whether to back up a file, just look at the file size without considering the file time

--modify-window=num the time stamp window that determines whether a file is in the same time, default is 0

-t--temp-dir=dir create temporary files in dir

--compare-dest=dir also compares files in DIR to determine whether a backup is required

-P equals to--partial

--progress Display backup process

-Z,--compress compression of backed-up files on transfer

--EXCLUDE=PATTERN Specifies to exclude file modes that do not require transfer

--INCLUDE=PATTERN Specifies the file mode that is not excluded and needs to be transferred

--exclude-from=file exclude files in the specified mode in file

--include-from=file does not exclude files that match the file-specified pattern

--version Print version Information

--address bound to a specific address

--CONFIG=FILE specifies a different profile and does not use the default rsyncd.conf file

--PORT=PORT Specifies the other rsync service ports

--blocking-io blocking IO for a remote shell

-stats gives the transfer status of some files

--progress transmission process in real time

--log-format=format Specify log file format

--password-file=file gets the password from file

--bwlimit=kbps limit I/O bandwidth, Kbytes per second

-H,--help display Help information