Centos6.6 configure rsync + sersync for real-time synchronization Distributed Multi-client distribution synchronization, rsyncsersync

Centos6.6 configure rsync + sersync for real-time synchronization Distributed Multi-client distribution synchronization, rsyncsersync

1. sersync project:

The sersync project uses inotify and rsync technology to synchronize server data to the solution in real time. inotify is used to monitor file system event changes on the server where sersync is located, rsync is a widely used local and remote data synchronization tool. It only operates on changed directory data.

Advantages of the sersync project:
1) write in C ++ to filter temporary files and repeated file operations generated by linux system files. When rsync is used together, network resources are reduced and the speed is faster.
2) It is easy to configure sersync. Download the source code at http://code.google.com/p/sersync/downloads/list. The bin directory contains compiled binary files, which can be used directly with the xml files in the bin directory.
3) use multiple threads for synchronization, especially when synchronizing large files, to ensure that multiple servers are synchronized in real time.
4) sersync provides an error handling mechanism to re-Synchronize failed files through the failure queue. If an error fails to be thrown, the files that fail to be synchronized are re-synchronized every 10 hours.
5) sersync comes with the crontab function. You only need to enable it in the xml configuration file to sort and synchronize data based on the pre-configuration.
6) sersync's built-in socket and http Protocol extensions can meet the special needs of removing the company's secondary development.

Inotify recognition event reference:
Http://www.ibm.com/developerworks/cn/linux/1-inotifynew
Http://blog.johntechinfo.com /? P = 122

2. Preparations before configuration
1) Environment preparation instructions:
192.168.233.129 rsync server (rsync server)
192.168.233.132 sersync

2) pre-configuration check
Cat/etc/redhat-release
CentOS release 6.6 (Final)
Uname-r
2.6.32-504. el6.x86 _ 64
Uname-m
X86_64

3) configure the rsync server
# Yum installation:
Yum install-y rsync
# It is easy to download, compile, and install. tar ==> configure -- ferfix =/usr/local/rsync ==> make & make install does not list the steps in detail, I chose YUM installation. After compilation and installation, pay attention to the PATH environment variables. The others are the same.
Wget http://pkgs.repoforge.org/rsync/rsync-3.0.9-2.el6.rfx.x86_64.rpm
# Check Installation
Rpm-qa rsync
Rsync-3.0.6-12.el6.x86_64
# Create a password file and modify permissions
Echo "rsyncback: 123.com">/etc/rsync. password
Chmod 600/etc/rsync. password

# Create a configuration file:
Cat>/etc/rsyncd. conf <EOF
#/Etc/rsyncd. conf
Uid = root
Gid = root
Use chroot = no
Max connections = 200
Hosts allow = *
Timeout = 600
Log file =/var/log/rsyncd. log
Pid file =/var/run/rsyncd. pid
Lock file =/var/run/rsync. lock
Ignore errors
Read only = false
List = false
Host deny = 0.0.0.0/32
Auth users = rsyncback
Secrets file =/etc/rsync. password
Transfer logging = yes
# Module definitions

[Data]
Comment = data by pjy
Path =/data/
####################################
[Www]
Comment = www by pjy
Path =/data/www/
###################################
[Bbs]
Comment = bbs by pjy
Path =/data/bbs/
##################################
[Blog]
Comment = blog by pjy
Path =/data/blog/
##################################
EOF

# Create a module monitoring directory. I use the root user, so I do not have the configuration permission. If the specified user is different, configure the user permission for the directory.
Mkdir-p/data/{www, bbs, blog}
Tree/data
/Data
── Bbs
── Blog
── Www
# Start the rsync service:
/Usr/bin/rsync -- damon
Ps-ef | grep rsync | grep-v grep
Root 7858 1 0? 00:00:00 rsync -- daemon
Netstat-lntp | grep 873
Tcp 0 0 0.0.0.0: 873 0.0.0.0: * LISTEN 7858/rsync
Tcp 0 0: 873: * LISTEN 7858/rsync
Lsof-I: 873
Command pid user fd type device size/OFF NODE NAME
Rsync 7858 root 3u IPv4 224418 0t0 TCP *: rsync (LISTEN)
Rsync 7858 root 5u IPv6 224419 0t0 TCP *: rsync (LISTEN

# Create a password file on sersync and test whether the synchronization is normal
Echo "123.com">/etc/rsync. password
Chmod 600/etc/rsync. password
Rsync-zvaP -- password-file =/etc/rsync. password rsyncback@192.168.233.129: data/
Tree data/
./
── Bbs
── Blog
── Www

3. Install the sersync Service
Wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/sersync/sersync2.5.4_64bit_binary_stable_final.tar.gz
Tar zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz
GNU-Linux-x86/
GNU-Linux-x86/sersync2
GNU-Linux-x86/confxml. xml
Cp-rf GNU-Linux-x86 // usr/local/sersync/
Cd/usr/local/sersync/
Mkdir bin logs conf
Mv confxml. xml conf/
Mv sersync2 bin/sersync
Tree
.
── Bin
│ ── Sersync
── Conf
│ ── Confxml. xml
── Logs
# Backing up configuration files
Cp conf/confxml. xml conf/confxml. xml. bak $ (date + % F)
# Configure Environment Variables
Echo "export PATH = $ PATH:/usr/local/sersync/bin">/etc/profile
Tail-1/etc/profile
Source/etc/profile
Which sersync
1) Single Instance Configuration:
# Modify the sersync configuration file: (you need to modify the remarks)
<? Xml version = "1.0" encoding = "ISO-8859-1"?>
<Head version = "2.5">
<Host hostip = "localhost" port = "8008"> <Debug start = "false"/>
<FileSystem xfs = "false"/>
<Filter start = "false">
<Exclude expression = "(. *) \. svn"> </exclude>
<Exclude expression = "(. *) \. gz"> </exclude>
<Exclude expression = "^ info/*"> </exclude>
<Exclude expression = "^ static/*"> </exclude>
<Delete start = "true"/>
<CreateFolder start = "true"/>
<CreateFile start = "false"/>
<CloseWrite start = "true"/>
<MoveFrom start = "true"/>
<Modify start = "false"/>
</Inotify>

<Sersync>
<Localpath watch = "/data"> # specify the local directory to be synchronized
<Remote ip = "192.168.233.129" name = "data"/> # specify the ip address and module name of the rsync server.
</Localpath>

<Rsync>
<CommonParams params = "-aruz"/> # specify the rsync synchronization Option
<Auth start = "true" users = "rsyncback" passwordfile = "/etc/rsync. password"/> # enable the specified password, corresponding to the user name and password file at the same time in rsync
# Example: rsync-zvaP -- password-file =/etc/rsync. password rsyncback@192.168.233.129: data/
<UserDefinedPort start = "false" port = "874" type = "codeph" text = "codeph"/> <! -- Port = 874 -->
<Timeout start = "false" time = "100"/> <! -- Timeout = 100 --> # specify timeout for 100 seconds
<Ssh start = "false"/>
</Rsync>

<FailLog path = "/usr/local/sersync/logs/www_rsync_fail_log.sh" # specify the storage location of the monitoring queue
TimeToExecute = "60"/> <! -- Default every 60 mins execute once -->
<Crontab start = "false" schedule = "600"> <! -- 600mins -->
<Crontabfilter start = "false">
<Exclude expression = "*. php"> </exclude>
<Exclude expression = "info/*"> </exclude>
</Crontabfilter>
</Crontab>
<Plugin start = "false" name = "command"/>
</Sersync>

<Plugin name = "command">
<Param prefix = "/bin/sh" suffix = "" ignoreError = "true"/> <! -- Prefix/opt/tongbu/mmm. sh suffix -->
<Filter start = "false">
<Include expression = "(. *) \. php"/>
<Include expression = "(. *) \. sh"/>
</Filter>
</Plugin>

<Plugin name = "socket">
<Localpath watch = "/opt/tongbu">
<Deshost ip = "192.168.138.20" port = "8009"/>
</Localpath>
</Plugin>
<Plugin name = "refreshCDN">
<Localpath watch = "/data0/htdocs/cms.xoyo.com/site/">
<Cdninfo domainname = "ccms.chinache.com" port = "80" username = "xxxx" passwd = "xxxx"/>
<Sendurl base = "http://pic.xoyo.com/cms"/>
<Regexurl regex = "false" match = "cms.xoyo.com/site ([/a-zA-Z0-9] *) .xoyo.com/images"/>
</Localpath>
</Plugin>
</Head>

# Configure English variables to prevent garbled characters:
LANG = EN
# Start the Monitoring Service (parameters are described later)
Sersync-r-d-o/usr/local/sersync/conf/confxml. xml

Set the system param
Execute: echo 50000000>/proc/sys/fs/inotify/max_user_watches
Execute: echo 327679>/proc/sys/fs/inotify/max_queued_events
Parse the command param
Option:-r rsync all the local files to the remote servers before the sersync work
Option:-d run as a daemon
Option:-o config xml name:/usr/local/sersync/conf/confxml. xml
Daemon thread num: 10
Parse xml config file
Host ip: localhosthost port: 8008
Daemon start, sersync run behind the console
Use rsync password-file:
User isrsyncback
Passwordfile is/etc/rsync. password
Config xml parse success
Please set/etc/rsyncd. conf max connections = 0 Manually
Sersync working thread 12 = 1 (primary thread) + 1 (fail retry thread) + 10 (daemon sub threads)
Max threads numbers is: 22 = 12 (Thread pool nums) + 10 (Sub threads)
Please according your cpu, use-n param to adjust the cpu rate
------------------------------------------
Rsync the directory recursivly to the remote servers once
Working please wait...
Execute command: cd/data/www & rsync-aruz-R -- delete. /rsyncback@192.168.233.129: www -- password-file =/etc/rsync. password>/dev/null 2> & 1
Run the sersync:
Watch path is:/data/www

# Monitoring of a single instance is completed. The test is successful!

2) Multi-instance configuration (this time, only one rsync server is used to simulate real-time synchronization configuration for multiple rsync servers. You only need to modify the IP address and module for multiple servers)
# Reliable 3 configuration files, simulating simultaneous distribution of www, bbs, and blog Code
Cd/usr/local/sersync/conf
Cp confxml. xml www_confxml.xml
Cp confxml. xml bbs_confxml.xml
Cp confxml. xml blog_confxml.xml

# Modify the configuration file: (only the modified parts are listed here)
Cat www_confxml.xml
<Sersync>
<Localpath watch = "/data/www">
<Remote ip = "192.168.233.129" name = "www"/>
</Localpath>

<Rsync>
<CommonParams params = "-aruz"/>
<Auth start = "true" users = "rsyncback" passwordfile = "/etc/rsync. password"/>
<UserDefinedPort start = "false" port = "874" type = "codeph" text = "codeph"/> <! -- Port = 874 -->
<Timeout start = "false" time = "100"/> <! -- Timeout = 100 -->
<Ssh start = "false"/>
</Rsync>

<FailLog path = "/usr/local/sersync/logs/www_rsync_fail_log.sh" timeToExecute = "60"/> <! -- Default every 60 mins execute once -->

Cat bbs_confxml.xml
<Sersync>
<Localpath watch = "/data/bbs">
<Remote ip = "192.168.233.129" name = "bbs"/>
</Localpath>

<Rsync>
<CommonParams params = "-aruz"/>
<Auth start = "true" users = "rsyncback" passwordfile = "/etc/rsync. password"/>
<UserDefinedPort start = "false" port = "874" type = "codeph" text = "codeph"/> <! -- Port = 874 -->
<Timeout start = "false" time = "100"/> <! -- Timeout = 100 -->
<Ssh start = "false"/>
</Rsync>

<FailLog path = "/usr/local/sersync/logs/bbs_rsync_fail_log.sh" timeToExecute = "60"/> <! -- Default every 60 mins execute once -->
Cat blog_confxml.xml
<Sersync>
<Localpath watch = "/data/blog">
<Remote ip = "192.168.233.129" name = "blog"/>
</Localpath>

<Rsync>
<CommonParams params = "-aruz"/>
<Auth start = "true" users = "rsyncback" passwordfile = "/etc/rsync. password"/>
<UserDefinedPort start = "false" port = "874" type = "codeph" text = "codeph"/> <! -- Port = 874 -->
<Timeout start = "false" time = "100"/> <! -- Timeout = 100 -->
<Ssh start = "false"/>
</Rsync>

<FailLog path = "/usr/local/sersync/logs/blog_rsync_fail_log.sh" timeToExecute = "60"/> <! -- Default every 60 mins execute once -->

# Modify the description: (Version: sersync2.5.4 _ 64)
1. Modify the default configuration of 23 to 28 rows, specify the local synchronization directory, peer ip address and module
2. Modify the default configuration of 30 to 33 rows, specify the parameters during rsync synchronization, enable the password file, specify the user and password file path.
3. Modify the 36 lines configured by default and specify the path of the Message Queue file.

# Restart the sersync service:
Ps-ef | grep sersync
Root 14270 1 0 21:44? At 00:00:00 sersync-r-d-o/usr/local/sersync/conf/confxml. xml
Root 14445 13674 0 00:00:00 pts/0 grep sersync
Pkill sersync
Ps-ef | grep sersync
Root 14459 13674 0 00:00:00 pts/0 grep sersync
Sersync-d-o/usr/local/sersync/conf/www_confxml.xml
Sersync-d-o/usr/local/sersync/conf/bbs_confxml.xml
Sersync-d-o/usr/local/sersync/conf/blog_confxml.xml
# View Processes
Ps-ef | grep sersync
Root 14464 1 0? At 00:00:00 sersync-d-o/usr/local/sersync/conf/www_confxml.xml
Root 14482 1 0? At 00:00:00 sersync-d-o/usr/local/sersync/conf/bbs_confxml.xml
Root 14500 1 0? At 00:00:00 sersync-d-o/usr/local/sersync/conf/blog_confxml.xml

# Test file synchronization
Cd/data
For I in 'LS'; do echo ddd >> I I/$ I. sh; done
Tree/
./
── Bbs
│ ── Bbsbbs. sh
│ ── Bbs. log
── Blog
│ ── Blogblog. sh
│ ── Blog. log
── Www
── Www. log
└ ── Wwwwww. sh
# Rsync server. The three modules are synchronized in real time!

# Add the sersync service to enable auto-start
Cat>/etc/rc. local <EOF
# Sync data to 192.168.233.192
Sersync-d-o/usr/local/sersync/conf/www_confxml.xml
Sersync-d-o/usr/local/sersync/conf/bbs_confxml.xml
Sersync-d-o/usr/local/sersync/conf/blog_confxml.xml
EOF

Tail-4/etc/rc. local
# Sync data to 192.168.233.192
Sersync-d-o/usr/local/sersync/conf/www_confxml.xml
Sersync-d-o/usr/local/sersync/conf/bbs_confxml.xml
Sersync-d-o/usr/local/sersync/conf/blog_confxml.xml

# Check the real-time monitoring process, which should be three sersync daemon processes,
Ps-ef | grep rsync
Root 14464 1 0? At 00:00:02 sersync-d-o/usr/local/sersync/conf/www_confxml.xml
Root 14482 1 0? At 00:00:02 sersync-d-o/usr/local/sersync/conf/bbs_confxml.xml
Root 14500 1 0? At 00:00:02 sersync-d-o/usr/local/sersync/conf/blog_confxml.xml

Parameter description:
-R performs an overall synchronization between the master server directory and the remote directory before enabling real-time monitoring. If a filter is set, the filter is true in the xml file, the-r parameter cannot be used for overall synchronization.
-O specifies the configuration file. If sersync is not specified, the default configuration file confxml. xml.
-N indicates the total number of threads in the default thread pool. If this parameter is not set, the default number of thread pools is 10.
-D. Start the service in the background.
-M does not synchronize, but only runs the plug-in. For example, sersync-m command does not synchronize the remote directory server after the event is monitored, but directly runs the command plug-in.

# Subsequent configuration file improvement updates .......