Rsync + inotify real-time synchronization, rsyncinotify Real-Time

Rsync + inotify real-time synchronization, rsyncinotify Real-Time

1. rsync

Compared with the traditional cp and tar backup methods, rsync has the advantages of high security, fast backup, and support for Incremental backup. rsync can meet the demand for data backup with low real-time requirements, for example, regularly back up the data on the file server to the remote server, and regularly perform data mirroring on the local disk.

As the scale of the application system continues to expand, it also puts forward higher requirements on data security and reliability, and rsync gradually exposes many shortcomings in high-end businesses. First, during rsync real-time synchronization, You need to scan all the files for comparison and perform the differential transmission. If the number of files reaches millions or even tens of millions, scanning all files is very time-consuming, and only a few files are being changed. This is a very inefficient method. Secondly, rsync cannot monitor and synchronize data in real time. Although it can start Synchronization Through the linux daemon, there is a time difference between the two departure actions, which leads to inconsistency between the server and the client, data cannot be completely restored when an application fails. For the above reason, rsync + inotify appears!

2. inotify (Monitoring)

 

Inotify is a powerful fine-grained and asynchronous file system event monitoring mechanism. Inotify can monitor the addition, deletion, modification, movement, and other minor events in the file system. With this kernel excuse, third-party software can monitor various changes in the file system, inotify-tools is such a third-party software.

 

1. server configuration (only rsync needs to be installed)

 

Share to/tmp:

 

1. Add a virtual user useradd rsync-s/sbin/nologin

 

Chown-R rsync. rsync/tmp/

 

2. Configure the rsyncd. conf configuration file

 

Vim rsyncd. conf

 

# Rsync_config ___________ start

 

# Rsyncd. conf start ##

 

Uid = rsync (User)

 

Gid = rsync (User)

 

Use chroot = no (to prevent security issues)

 

Max connections = 200 (how many clients can connect to my backup server)

 

Timeout = 300 (timeout, how long will the connection be disconnected without action)

 

Pid file =/var/run/rsyncd. pid (process number, put the process number in this file)

 

Lock file =/var/run/rsync. lock (equivalent to the concept of "lock", the concept of a toilet lock)

 

Log file =/var/log/rsyncd. log (if an error occurs, you can view the log file)

 

[Tmp] (module)

 

Path =/tmp/(path)

 

Ignore errors (ignore error)

 

Read only = false (read-only is false, readable, and writable)

 

List = false (list not allowed)

 

Hosts allow = 10.0.0.0/24 (allowed hosts)

 

Hosts deny = 0.0.0.0/32 (rejected host)

 

Auth users = rsync_backup (supports Virtual users)

 

Secrets file =/etc/rsync. password (password file of the user)

 

# Rsync_config ___________ end

 

3. Create a password file

 

Echo "rsync_backup: 123456">/etc/rsync. password

 

All password files have the 600 permission chmod 600/etc/rsync. password

 

4. rsync -- daemon mode startup

 

5. Add rsync -- daemon to/etc/rc. local.

 

Echo "/usr/bin/rsync -- daemom">/etc/rc. local

 

 

 

Ii. Client Configuration

 

Install rsync and inotify

 

1. Install rsync (install yum)

 

2. Create a password authentication File

 

Echo "123456">/etc/rsync. password only requires a password

 

Set the permission to 600 chmod 600/etc/rsync. password

 

3. Install inotify:

 

Cd/home/cai/tools/

 

Wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-too

 

 

 

54 tar xf inotify-tools-3.14.tar.gz

 

55 ls

 

56 cd inotify-tools-3.14

 

57./configure -- prefix =/usr/local/inotify-tools-3.14

 

58 make & make install

 

59 yum install-y gcc

 

60./configure -- prefix =/usr/local/inotify-tools-3.14

 

61 make & make install

 

62 cd/usr/local/inotify-tools-3.14/

 

63 ls

 

64 ln-s/usr/local/inotify-tools-3.14 // usr/local/inotify

 

 

 

4. All scripts are stored in/server/scripts.

 

Vim/server/scripts/rsync. sh

 

#! /Bin/sh

 

Host = 192.168.76.129

 

Src =/tmp/

 

Des = tmp

 

User = rsync_backup

 

/Usr/local/inotify/bin/inotifywait-mrq -- timefmt '% d % m % y % H: % m' -- format' % T % w % f % e'-e modify, delete, create, attrib $ src | \

 

While read files

 

Do

 

/Usr/bin/rsync-avz -- delete $ src $ user @ $ host: $ des -- password-file =/etc/rsync. password

 

Echo "$ {files} was rsynced">/var/log/rsyncd. log 2> & 1

 

Done

 

Exit o

 

~

 

And grant the 764 permission

 

Test Script: sh-x/server/scripts/rsync. sh

 

Run the script sh/server/scripts/rsync. sh &

 

Put the rsync. sh script into the boot start item: echo "/tmp/rsync. sh">/etc/rc. local