Rsync+inotify Real-time backup

About Rsync

Rsync has four application modes 1, Shell local mode 2, remote Shellmoshi 3, Query mode 4, Server mode

1. Local mode is used to replicate directories to another directory Rsync-av aa/bb

2. Remote Shell mode copy the local directory to another system Rsync-av AA 192.168.0.10:BB

3. Query (list) mode view the contents of a directory in a remote system rsync-a 192.168.0.10:BB

4. Server mode enables a daemon to receive or send files based on the C/S mode server

Server address

Server 192.168.0.101/web/webdata

Web1 192.168.0.103/web1/webdata

WEB2 192.168.0.104/web2/webdata

WEB3 192.168.0.106/web3/webdata

1. Download the relevant software package

wget http://rsync.samba.org/ftp/rsync/src/rsync-3.1.1.tar.gz

wget https://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

Configuring rsync on the server

Touch/etc/rsyncd.conf

UID = nobody

GID = Nobody

Use chroot = no

Max connections = 10

Strict modes = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsync.lock

Log file =/var/log/rsyncd.log

[web]

Path =/web/webdata/

Comment = Ixdba file

Ignore errors

Read Only = no

Write only = no

Hosts allow = *

List = False

UID = root

GID = root

Auth users = WebUser

Secrets file =/etc/server.pass

The meaning of each parameter

User ID that the UID daemon has

The group ID that the GID daemon has

Max connections Maximum concurrent connections default 0 means no Limit

Strict modes Check the permissions of the file Yes to check password file permissions password file permissions must be root user rights

PID File PID files path

Lock file supports Max connections locks files

Log file logfile Path

[WebUser] Module name

File or directory path backed up by path

Ignore errors means ignoring I/O errors

Read only set to no user can upload file Yes indicates read only

Write only set to no user can download the file Yes indicates that it cannot be downloaded

Hosts allowed by hosts allow

Hosts deny denied connections to host

List when a customer requests a list of available modules, whether the module is listed by default to True to hide how I set to False

Auth users can connect to the module's user name

Secerts file User: path to password files

Create a password file

echo "Backup" >>/etc/server.pass

Note: The password file on the Publisher only sets the password file on the node server to set the account: password.

To start the rsync daemon on the server system

/usr/local/bin/rsync--daemon

echo "/usr/local/bin/rsync--daemon" >>/etc/rc.local add rsync service to the self-boot file

Installing the Rsync service on web123 servers separately

Install Inotify-tools at the Publisher (node server does not need to be installed)

Detects if the kernel supports less than 2.6.13 and needs to recompile the kernel to add support for inotify

Uname-r

[Email protected] ~]# ll/proc/sys/fs/inotify

Total dosage 0

-rw-r--r--1 root root 0 February 10:25 max_queued_events

-rw-r--r--1 root root 0 February 10:25 max_user_instances

-rw-r--r--1 root root 0 February 10:25 max_user_watches

If the above three outputs indicate that the system supports the installation under INotify

TAR-ZXVF inotify-tools-3.13.tar.gz

CD inotify-tools-3.13

./configure

Make

Make install

ll/usr/local/bin/inotifywa*

-rwxr-xr-x 1 root root 32030 February 10:39/usr/local/bin/inotifywait

-rwxr-xr-x 1 root root 33979 February 10:39/usr/local/bin/inotifywatch

Inotifywait and Inotifywatch two files are generated after installation, inotifywait is used to wait for a specific time on a file or set of files, and can monitor file and directory settings inotifywatch used to collect monitored file system statistics

Configuring Rsync on a three-node server

Web1

UID = nobody

GID = Nobody

Use chroot = no

Max connections = 10

Strict modes = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsync.lock

Log file =/var/log/rsyncd.log

[Web1]

Path =/web1/webdata

Comment = Web1 file

Ignore errors

Read Only = no

Write only = no

Hosts allow = *

List = False

UID = root

GID = root

Auth users = Web1user

Secrets file =/etc/server.pass

Web2

UID = nobody

GID = Nobody

Use chroot = no

Max connections = 10

Strict modes = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsync.lock

Log file =/var/log/rsyncd.log

[WEB2]

Path =/web2/webdata

Comment = Ixdba file

Ignore errors

Read Only = no

Write only = no

Hosts allow = *

List = False

UID = root

GID = root

Auth users = Web2user

Secrets file =/etc/server.pass

Web3

UID = nobody

GID = Nobody

Use chroot = no

Max connections = 10

Strict modes = yes

PID file =/var/run/rsyncd.pid

Lock file =/var/run/rsync.lock

Log file =/var/log/rsyncd.log

[WEB3]

Path =/web3/webdata

Comment = Ixdba file

Ignore errors

Read Only = no

Write only = no

Hosts allow = *

List = False

UID = root

GID = root

Auth users = Web3user

Secrets file =/etc/server.pass

After you configure the rsyncd.conf of the three node servers, create the password files separately. Start the rsync daemon in turn and add the service to the self-boot file

echo "/usr/local/bin/rsync--daemon" >>/etc/rc.local

Configure the Server Content publishing node

Publish your content by writing shell scripts

#!/bin/bash

host1=192.168.0.103

host2=192.168.0.104

host3=192.168.0.106

src=/web/webdata/

Dst1=web1

Dst2=web2

Dst3=web3

User1=web1user

User2=web2user

User3=web3user

/usr/local/inotify/bin/inotifywait-mrq--timefmt '%d/%m/%y%h:%m '--format '%T%w%f%e '-e modify,delete,create,attrib $ src | While read files

Do

/USR/LOCAL/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/etc/server.pass $src [email protected] $host 1::$ Dst1

/USR/LOCAL/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/etc/server.pass $src [email protected] $host 2::$ Dst2

/USR/LOCAL/BIN/RSYNC-VZRTOPG--delete--progress--password-file=/etc/server.pass $src [email protected] $host 3::$ Dst3

echo "${files} was rsynced" >>/tmp/rsynclog 2>&1

Done

Put this script in the background to run and add to the self-boot file

Sh/root/rsync.sh &

echo "/root/rsync.sh &" >>/etc/rc.local

The relevant parameters in the script are as follows

--TIMEFMT the output format for the specified time

--format Specify the details of the change file

Sometimes this happens when a large file is written to the directory of the publisher, and when it takes a while to write the large file, the inotify will continuously output the updated information of the file so that it will continue to go out of sync and consume a lot of system resources, The ideal penalty is to wait for the write-up after the rsync synchronization is triggered, in which case the INotify monitoring event can be modified as-e Close_write,delete,create,attrib

Test Rsync+inotify Real-time synchronization

Add or remove files or directories from the server server's/web/webdata/directory to see if the node server is updated.

Rsync+inotify Real-time backup