Server Data synchronization Via Linux rsync, rsync+inotify between Load Balance Server

Directory

1 application Scenarios for remote file synchronization 2. rsync+crontab3. Rsync+inotify

1. Application Scenarios for remote file synchronization

In a load-balanced cluster scenario, there is often a load Balance Server on the front end of multiple Web servers that provides

1 The client accesses a unified domain name to this load Balance, for example: Www.test.com,Load Balance Server allocates the request reasonably to the child nodes of the cluster based on the load scheduling algorithm 2. The files uploaded by the client are also assigned" randomly " by the load Balance server to a node server in the cluster.

This creates a problem, how to ensure the status of each sub-node server, data consistency, that is, we ensure that the client can be treated as single point as the upload, and download the file, which requires a two-way data synchronization between the various sub-nodes of the cluster

2. Rsync+crontab

rsync (remote sync) is a data mirroring Backup tool under Unix-like systems. A fast incremental backup tool remote sync that supports local replication or synchronizes with other SSH and rsync hosts

0x1:rsync Features

 1  . can be mirrored to save the entire directory tree and file system  2  . It is easy to keep the original file permissions, time, soft and hard links  3  Span style= "color: #000000;" >. Install  4   quick  1  ) when the first synchronization, rsync copies everything, but the next time only the modified file, incremental backup  2  ) rsync can compress and decompress during data transfer, so it can use less bandwidth  5   1   2  ) can be connected via a direct socket  6 . Support for anonymous transmission for Web site mirror image 

After understanding the basic concepts of rsync, let's learn how to configure the Rsync service, in essence, Rsync is a peer-to architecture service system, so if we want to make two-way backup between multiple servers, then each server is server and clinet

0x2:rsync Service Installation Configuration

1. Deploying the rsync service side

1. Install the Super daemon Yum–y install xinetd2. Provisioning profiles for rsync services vim/etc/rsyncd.conf #Global Settings uid=Nobodygid=Nobodyuse chroot=Nomax Connections=5Timeout= -pid File= /var/run/Rsyncd.pidlockfile= /var/run/rsyncd.Locklog File= /var/log/rsyncd.log #module [Zhenghan]path=/opt/backup/Ignore Errors=Yesread only=Nowrite only=nohosts Allow= *hosts Deny= *List=Yesuid=Rootgid=Rootauth Users=zhenghansecrets File=/etc/zhenghan.passwd3Configure the Rsync authentication file (no need to configure if no password is required) Vim/etc/Zhenghan.passwdecho"Zhenghan:zhenghan">/etc/zhenghan.passwd Cat/etc/Zhenghan.passwdzhenghan:zhenghan4. Modify/etc/the CHANDAO.PASSWD has a permission of 600chmod -/etc/zhenghan.passwd5. Configure service start Chkconfig rsync onservice xinetd start6Check if Port 873 successfully listens to SS-ANTLP | Grep8737. Check the rsync running status Chkconfig--list

2. Deploying the Rsync client

1. Set the password file for the rsync client1) password file echo Zhenghan>/etc/zhenghan.passwd2) The password file for the database echo Zhenghan>/etc/db_zhenghan.passwd2The permission to modify the password file is 600chmod -/etc/Zhenghan.passwdchmod -/etc/db_zhenghan.passwd3test whether rsync client can pull successfully from Rsync server-VZRTOPG--password-file=/etc/zhenghan.passwd [email protected]192.168.200.16:: Zhenghan/* /tmp4. rsync sync script #!/bin/bash#func:rsync for Zhenghan #Version: v1.0 ntpdate 192.168.200.16 >/dev/null 2>&1date= ' date +%y-%m-%d ' dest_chandao=/opt/backup/zhenghan/testdest_db=/opt/backup/zhenghan/db_zhenghanremote_zhenghan= 192.168.200.16remote_db=192.168.200.18module_zhenghan=zhenghanmodule_db=db_zhenghan[email protected] Main ( {RSYNC-VZRTOPG--password-file=/etc/zhenghan.passwd [email protected] $remote _zhenghan:: $module _zhenghan/*$ date* $dest _zhenghan If [$?-eq 0];then echo "zhenghan-* $date * was rsynced" >>/opt/log/zhenghan_rsyn     C.log 2>&1;         else echo "zhenghan-* $date * is not rsynced" >>/opt/log/zhenghan_rsync.log 2>&1; echo "Zhenghan rsync was Faild" |     Mail-s "Zhenghan backup" $user 2; Fi RSYNC-VZRTOPG--password-file=/etc/db_zhenghan.passwd [email protected] $remote _db:: $module _db/* $date * $ dest_db if [$?-eq 0];then echo "db_zhenghan_* $date * was rsynced" >>/OPt/log/zhenghan_rsync.log 2>&1;         else echo "db_zhenghan_* $date * is not rsynced" >>/opt/log/zhenghan_rsync.log 2>&1; echo "Db_zhenghan rsync was Faild" |     Mail-s "Zhenghan backup" $user 2; Fi}main

3. Scheduled Tasks

Rsync is one-to-peer architecture where data synchronization between server and client can be done in two directions, and after the service side and clients are configured, it is common practice to synchronize data between client and server using the Crontab timer task

Crontab- for zhenghan * * * */BIN/SH/OPT/SCRIPTS/RS_ zhenghan.sh

Relevant Link:

http://Baike.baidu.com/link?url=edzps6hvi-m9hmocs5bbx9arl2pmj1v6ughwecdjp7yz_ 40MNTLL0WRIMMPQOOBQPL6OHFVTYZMWGJF8XE_I3Ahttp://nmshuishui.blog.51cto.com/1850554/1532927 

3. rsync+inotify

0x1: Advantages of rsync+inotify

1in the case of a very large amount of data, the traditional task plan, rsync will first go through the target directory, all the data to do a signature comparison, and then carry out the differential transmission, this process will be very long, for those who require real-time update relatively high enterprise, is undoubtedly a nightmare2. Inotify is a powerful, granular, asynchronous file system event monitoring mechanism, from the Linux kernel to 2.6.13, joined the INotify support, through the inotify can monitor the file system Add, * *, modify, mobile and other subtle events, using this kernel interface, third-party software can monitor the file system under the various changes, and inotify-That's how tools are. A third-party software3. Server Performance1) rsync+Crontab will periodically check for file updates, which is bound to cause server performance degradation2) rsync+The inotify combination is a triggered update that is updated only when there are changes to the data file, thus improving server performance relative to the former4. Data real-time1) rsync+Crontab is a recurring task plan that does not guarantee the real-time data2Rsync+inotify combination is triggered update, as long as there is data changes, immediately synchronize the update

0x2:rsync+inotify installation Configuration

The biggest difference between rsync+inotify and Rsync+crontab is the trigger condition of synchronization, that is, the initiation condition of synchronization command is different, rsync+inotify is to trigger bidirectional file synchronization in real time through the inotify mechanism of Linux.

Relevant Link:

http://nmshuishui.blog.51cto.com/1850554/1387048

Copyright (c) Littlehann All rights reserved

Server Data synchronization Via Linux rsync, rsync+inotify between Load Balance Server