RSYNC+LSYNCD Real-time synchronization of (local and remote) files

LSYNCD is based on the Lua language development, integrates the rsync and notify  implementation files real-time synchronization system Environment  cat /etc/issueCentOS release 6.6  ( Final)  uname -srlinux 2.6.32-504.el6.x86_64 Server Planning rsync server:                    192.168.10.241rsync +  LSYNCD Server  :          192.168.10.231. Configuring Rsync server 1. Configure XINETD to manage rsync and edit the configuration file  yum install xinetd -y vim / etc/xinetd.d/ Rsync-------------------------------------------------------------------------------------------------------------------- ----------service rsync{        disable = no         socket_type     = stream         wait             = no        user            =  root        server           = /usr/bin/rsync        server_args      = --daemon        log_on_failure  +=  USERID}-------------------------------------------------------------------------------------------------------- ---------------------- rpm -qa | grep rsyncrsync-3.0.6-6.el5_112. Providing configuration file global options for rsync  strict modes =yes   whether to check permissions for password files  port = 873  default port 873 log  File = /var/log/rsyncd.log Log record file  pid file = /var/run/rsyncd.pid  run process ID where to write  [home]path = /opt/  Here is the authentication module name, which needs to be specified on the client side  max connections = 0  Client maximum connections, default 0 (No limit)  uid = root  Specifies the uid gid = root  that the daemon should have when the module transmits files specifies the gid  that the daemon should have when the module transfers files ignore errors  can ignore some unrelated IO errors  read only = no no client can upload files, yes read-only  write  only = no no client can download files, yes cannot download  hosts allow = *   allow any host connection  hosts deny = 10.5.3.1  prohibit the specified host connection  auth users = root  authenticated user name, If this is not the case, it is anonymous  secrets file = /etc/backserver.pas  specify the authentication password file location  -------------------- ----------------------------------------------------------------------------------------------------------3. Add the authentication file and modify the permissions to 600# echo  "root:123456"  > /etc/backserver.pas# chmod 600   /ETC/BACKSERVER.PAS LL /ETC/BACKSERVER.PAS-RW------- 1 root root 11  Jul 27 15:22 /etc/backserver.pas  4. Starting the rsync server # /etc/init.d/xinetd  Startstarting xinetd:                                             [  ok  ][[email protected] ~]# netstat - ntl | grep 873tcp        0       0 0.0.0.0:873                  0.0.0.0:*                    listen two. Client Configuration LSYNCD, real-time synchronization based on LSYNCD (less latency) 1.1 installation lsyncd yum  search lsyncd  (source server installation)   If this package is not found, only that you do not have the Yum expansion pack installed for CentOS, you must first install the expansion pack  rpm -ivh  After the installation of the expansion pack is complete, you can install LSYNCD  yum install lsyncd -y1.2 compile the installation Lsyncd yum install lua  lua-devel -y wget -c  tar xf release-2.1.5 cd lsyncd-2.1.5 ./configure make  && make install2. Launch Script  vim /etc/init.d/ LSYNCD------------------------------------------------------------------------------------------------------------------- -----------#!/bin/bash## chkconfig: - 85 15# description: lightweight  inotify based sync daemon. /etc/init.d/functions. /etc/sysconfig/network[  "$ NETWORKING " = " no " ] && exit 0lsyncd_options="-pidfile /var/run/ Lsyncd.pid /etc/lsyncd.conf "retval=0prog=" LSYNCD "Thelock=/var/lock/subsys/lsyncdstart ()  {     [ -f /etc/lsyncd.conf ] | |  exit 6        echo -n $ "Starting  $prog:   "        if [  $UID  -ne 0 ]; then& nbsp;               retval=1                 failure         else                 daemon /usr/local/bin/lsyncd  $LSYNCD _options                 RETVAL=$?                 [  $RETVAL  -eq 0 ]  && touch  $thelock         fi;         echo        return  $RETVAL}stop ()  {        echo -n $ "stopping  $prog: "          if [  $UID  -ne 0 ]; then                 RETVAL=1                 failure        else                 killproc  lsyncd                retval=$ ?                 [ $ retval -eq 0 ] && rm -f  $thelock          fi;        echo         return  $RETVAL}reload () {        echo -n $ " reloading  $prog:  "        killproc lsyncd -HUP         RETVAL=$?        echo         return  $RETVAL}restart () {        stop         start}condrestart () {    [ -e  $thelock  ] && restart    return 0}case  "$"  in   Start)         start        ;;   stop)         stop         ;;   restart)         restart         ;;   reload)         reload        ;;   condrestart)         condrestart         ;;   status)         status lsyncd         RETVAL=$?        ;;   *)         echo $ "usage: $0 {start|stop| Status|restart|condrestart|reload} "        retval=1esacexit $ RETVAL------------------------------------------------------------------------------------------------------------------- -----------3. Edit the Rsync client, provide the add password file, and modify the permissions to 600 echo  "123456"  > /etc/backserver.pas  CHMOD 600 /ETC/BACKSERVER.PAS LL /ETC/BACKSERVER.PAS-RW------- 1 root root  7 jul 27 15:34 /etc/backserver.pas4. Editing the LSYNCD configuration file  vim /etc/lsyncd.conf------------------------------------------------------------------------------------------------------------------- -----------settings {   logfile    =  "/tmp/lsyncd.log",    statusFile =  "/tmp/lsyncd.status",   statusinterval = 5,    nodaemon = true, }  Local sync 1.1  local directory sync: DIRECT:CP/RM/MV.   Apply: The document of the sync {    default.direct,    source , the change is not big    =  "/opt/src",    target    =  "/opt/dest",     delay = 1,    maxprocesses = 1,  } 1.2  Local Directory Sync rsync mode: rsyncsync{    default.rsync,    source =   "/opt/",    target =  "/home/mogilefs/",     delete  = true,    exclude={  "Test"  },    rsync = {         compress = true,        verbose = true,         archive  = true,    }} Remote Sync 2.1  remote sync:  rsync mode  + rsyncd daemonsync {    default.rsync ,    source    =  "/opt/back3/",     target     =  "[Email protected]::home",     delete = true,     exclude = {  ". tmp"  },    delay = 1,     rsync = {        binary =  "/usr/bin/rsync",        archive = true,         compress = true,        verbose  = true,         password_file =  "/etc/backserver.pas",         _extra    = {"--bwlimit=200"},         }    } 2.2  Remote directory synchronization: rsync mode  + ssh shellsync {    default.rsync,    source =  "/opt/back/",     target =  "[email protected]:/opt",     delete = true,     -- exclude={  "Test"  },    rsync = {         binary =  "/usr/bin/rsync",         compress = true,        verbose =  true,        archive = true,         rsh =  "/usr/bin/ssh -p 22 -o stricthostkeychecking=no"     }}  3  Remote directory Sync: rsync mode  + rsyncssh with the same sync {    default.rsyncssh as above     source    =  "/opt/back2/",    host       =  "192.168.10.241",    targetdir =  "/opt/",--   excludeFrom =  "/etc/rsyncd.d/rsync_exclude.lst",     maxdelays =  1,    delay = 1,    rsync = {         binary =  "/usr/bin/rsync",         archive = true,        compress = true ,        verbose   = true,         _extra = {"--bwlimit=2000"},        },     ssh  = {        port  =   22        }    }---------------------------- --------------------------------------------------------------------------------------------------5. Key-Free Login ( SSH-based synchronization requires SSH password-free login on a server that is synced remotely  ssh-copy-id -i  ~/.ssh/id_rsa.pub [email  Protected] 6. Start the log file for the RLSYNCD service  /ETC/INIT.D/LSYNCD START7.LSYNCD and the status file  tailf /tmp/ lsyncd.log tue jul 28 12:12:15 2015 normal: recursive startup  rsync: /opt/back3/ -> [email protected]::home/tue jul 28 12:12:19  2015 normal: startup of  " /opt/back3/" finished. tue jul 28 12:13:07 2015 normal: recursive startup rsync: /opt/ Back3/ -> [email protected]::home/tue jul 28 12:13:12 2015 error:  Temporary or permanent failure on startup of  "/opt/back3/" .  terminating since  "insist"  is not set. tue jul 28 12:13:14 2015 normal: recursive startup rsync: /opt/ Back3/ -> [email protected]::home/tue jul 28 12:13:18 2015 error:  Temporary or permanent failure on startup of  "/opt/back3/" .  terminating since  "insist"  is not set. tue jul 28 12:13:35 2015 normal: recursive startup rsync: /opt/ Back3/ -> [email protected]::home/tue jul 28 12:13:38 2015 normal: startup of  "/opt/back3/"  finished. tue jul 28 12:15:50 2015 normal: recursive startup rsync: /opt/ Back3/ -> [email protected]::home/tue jul 28 12:15:53 2015 normal:  Startup of  "/opt/back3/"  finished.tailf /tmp/lsyncd.statussync1 source=/opt/ Back3/there are 1 delaysactive init excluding:  nothing. inotify watching 1 directories  1: /opt/back3/

lsyncd.conf Main configuration Options Description:

Settings configuration section:
Global Settings--start with comments, here are a few common options descriptions:
LogFile Defining log files
Stausfile Defining State files
Nodaemon=true indicates that the daemon mode is not enabled, the default
Statusinterval writes the LSYNCD state to the above statusfile interval, which defaults to 10 seconds
inotifymode specifies inotify monitored events, default is Closewrite, and can be Modify or closewrite or Modify
The maximum number of maxprocesses synchronization processes. If there are 20 simultaneous files that need to be synchronized, and maxprocesses = 8, you can see that there are 8 RYSNC processes
maxdelays how many monitored events are activated to activate one synchronization, even if the delay latency is not yet


Sync configuration section:
Define the synchronization parameters, and you can continue to use Maxdelays to override the settings global variables.
The general first parameter specifies what mode LSYNCD operates in: rsync, RSYNCSSH, direct three modes:
Default.rsync: Synchronization between local directories, using rsync, can also achieve the use of SSH remote rsync effect, or daemon way to connect remote RSYNCD process;
Default.direct: Synchronization between local directories, using CP, RM and other commands to complete the differential file backup;
DEFAULT.RSYNCSSH: Sync to remote host directory, rsync SSH mode, need to use key to authenticate
Source directory to synchronize with, using absolute path.
Target defines the destination address.
An example of three modes is given later.
Init This is an optimization option, when init = False, only the synchronization process starts after the Change event file, the original directory even if there are differences will not be synchronized. The default is True
Delay cumulative event, waiting for rsync synchronization delay time, default 15 seconds (maximum cumulative to 1000 non-consolidated events). That is, 15s within the monitoring directory changes, will accumulate to one time rsync synchronization, avoid too frequent synchronization. (can be combined means that the same file was modified two times in 15s, and only the latest files were synchronized)
Excludefrom The Exclude option, specify the excluded list file later, and if it is a simple exclusion, you can use exclude = {"FILENAME"}.
The exclusionary rule here is a little different from the native rsync, which is simpler:
Any part of the monitoring path that matches to a text will be excluded, for example/var/web/test can match the rule web
If the rule starts with a slash/start, it starts from scratch to match all
If the rule ends with a/end, match the end of the monitoring path
Match any character, but not include/
* Matches 0 or more characters, but does not include/
* * Matches 0 or more characters, can be/
Delete to keep target in full sync with souce, lsyncd default Deletes = True to allow simultaneous deletion. Besides false, it also has startup, running value.
Rsync: (Note that delete and exclude are inherently rsync options, which are configured in sync, and I want to do this to reduce the cost of rsync)
Bwlimit speed limit, unit kb/s, same as rsync (such an important option is not marked in the document)
Compress compression transport defaults to TRUE. On the tradeoff between bandwidth and CPU load, local directory synchronization can consider setting it to false
Perms default reserved file permissions.

Other options for Rsync
Other configuration items that are unique to the RSYNCSSH mode, such as host, TARGETDIR, Rsync_path, Password_file, are also available.
Lsyncd.conf can have multiple sync, the respective source, the respective target, the respective mode, and not affect each other.

Use the command to load the configuration file, start the daemon, and automatically synchronize the directory operation.
Lsyncd-log exec/usr/local/lsyncd-2.1.5/etc/lsyncd.conf

Note: The 1.LSYNCD version:2.1.4 version has a bug, does not support Rsync's password_file parameter, LSYNCD version:2.1.5 has been repaired, and allows the rsync client to have write permissions. 2.rsyncops={"-avz", "--delete"} is not supported in the 2.1.* version.

This article is from the Linux tours blog, so be sure to keep this source http://openlinuxfly.blog.51cto.com/7120723/1679279

RSYNC+LSYNCD Real-time synchronization of (local and remote) files