Rsync+inotify Implementing Synchronous data

1. rsync

Man to sync, you'll find: Sync-flush file system buffers, which is a command to synchronize the data in the buffer to the filesystem, and Rsync is the remote rsync, which is a long-distance synchronization tool, With the CP and SCP functions, the rsync command is almost identical to the SCP. Rsync is triggered synchronously by the Super Daemon xinetd.

1) Advantages: Rsync is faster, more secure, and supports incremental backups than CP and SCP. Rsync in the process of data synchronization, not like CP all copy, but first to compare to the signature, only a different copy, if the same, do not need to do extra operations. By using Rsync+crontab, you can address scenarios where real-time requirements are not too high.

2) Cons: In this big data age, if the amount of data is very large, every time you do a task plan, rsync will first go through the target directory, all the data to do a feature code comparison, and then carry out the differential transmission, this process will be very long, for those who require real-time update of the higher enterprise, is undoubtedly a nightmare. The rsync+inotify combination is able to monitor our file system in real time and only triggers it to sync when the data changes!

Rsync has 4 different modes

1. Local mode.

2. Remote Shell mode

3. Query mode

4.c/s mode

2, INotify

Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, the Linux kernel from 2.6.13, joined the Inotify support, through Inotify can monitor the file system to add, delete, modify, move and other subtle events, using this kernel interface, Third-party software can monitor the file system under the various changes, and inotify-tools is such a third-party software.

3. Rsync+inotify Benefits

1) Server performance: Rsync+crontab will periodically check whether there are file updates, which will inevitably cause server performance degradation, and the rsync+inotify combination is triggered update, only when the data file changes, will be updated, so the former, is to improve the server performance.

2) Data Real-time: Rsync+crontab is a periodic task plan, can not guarantee the real-time data; rsync+inotify combination is triggered update, as long as there is data changes, immediately synchronize updates.

The rsync command is a fast, multi-use, remote and local file copying tool;

Command syntax format:

Local:rsync [OPTION ...] Src... [DEST]

Access via remote shell:

Pull:rsync [OPTION ...] [[email protected]] Host:src ... [DEST]

Push:rsync [OPTION ...] Src... [[email protected]] Host:dest

Access via rsync daemon:

Pull:rsync [OPTION ...] [[email protected]] Host::src ... [DEST]

rsync [OPTION ...] rsync://[[email protected]]host[:P ort]/src ... [DEST]

Push:rsync [OPTION ...] Src... [[email protected]] HOST::D EST

rsync [OPTION ...] SRC ... rsync://[[email protected]]host[:P ort]/dest

Options:

-V,--verbose: verbose information;

-R,--recursive: recursive to the directory;

1. Local mode

RSYNC-RV/ETC/PASSWD./

Copy passwd to current directory

2. Remote Shell mode

RSYNC-RV/ETC/PASSWD 172.18.11.12:/tmp

Copy the passwd to the remote host, similar to the SCP command requires a user password;

3. List mode

Rsync-r 172.18.11.12:/tmp

View files in remote directory

The focus is on the fourth mode, which enables data synchronization:

4.c/s mode, data synchronization:

Lab Environment:

master:172.18.11.11

slave:172.18.11.113

Experimental objectives:

Implementation of the master host in the /var/ftp/directory as long as there are file changes (can be defined), will be synchronized to the slave host in real-time /home/ftp directory.

On the slave side:

]# vim/etc/rsyncd.conf

650) this.width=650; "src=" http://s5.51cto.com/wyfs02/M00/80/51/wKiom1c9qEzByHpkAABcd49QHok057.png "title=" Image 2. PNG "alt=" Wkiom1c9qezbyhpkaabcd49qhok057.png "/>

Create a password file/etc/rsyncd.pass the following format and ensure that the permissions are 600 or 400

]# Vim/etc/rsyncd.pass

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/80/4E/wKioL1c9qZyg9OZjAAAMZbeM8co553.png "title=" Image 3. PNG "alt=" Wkiol1c9qzyg9ozjaaamzbem8co553.png "/>

]# chmod 600/etc/rsyncd.pass

Start the daemon process

]# rsync--daemon

]# Netstat-tnlp|grep rsync

View 873-port monitoring;

On the master side:

]# Vim /etc/rsyncd.pass

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/80/51/wKiom1c9qSrxOfJ5AAAKn5vwmq8889.png "title=" Image 4. PNG "alt=" Wkiom1c9qsrxofj5aaakn5vwmq8889.png "/>

Note: This password file only enter the password set on the slave side;

manual test, master side passwd file whether it can sync to the directory where the slave end is set;

]# rsync-avz/var/ftp/pub/passwd [email protected]:: FTP--password-file=/etc/rsyncd.pass

Explain:

Root is the user specified for the slave end;

172.18.11.113 for slave end address;

/etc/rsyncd.pass the password of the specified user for storing the slave end;

In the slave side of the synchronization directory to see if you receive passwd files;

Installing Inotify-tools

]# yum-y Install Inotify-tools

Write the monitoring script and load it into the background execution:

]# vim /root/rsync.sh

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/80/51/wKiom1c9rBHCR25JAABuhoilF7o364.png "title=" image 5. PNG "alt=" Wkiom1c9rbhcr25jaabuhoilf7o364.png "/>

Add script to Background execution

]# SH inotify.sh &

Real-time sync test :

On the master side:

]# for a in ' seq;d o touch $a;d one

Creation of 200 files;

]# ll--time-style=full-iso

650) this.width=650; "src=" http://s1.51cto.com/wyfs02/M01/80/51/wKiom1c9rzzg3yuJAACvkozzoYE307.png "title=" image 6. PNG "alt=" Wkiom1c9rzzg3yujaacvkozzoye307.png "/>

And then to the slave end view:

]# ll --time-style=full-iso/home/ftp

650) this.width=650; "src=" http://s2.51cto.com/wyfs02/M01/80/4F/wKioL1c9sKCDKy63AADiw2jNsIM594.png "title=" Image 8. PNG "alt=" Wkiol1c9skcdky63aadiw2jnsim594.png "/>

The real-time data synchronization is realized.

Rsync+inotify Implementing Synchronous data