How to use Sersync to implement multiple server real-time synchronization files

I. Introduction of SERSYNC Project

Project Address: http://code.google.com/p/sersync/

We often use Rsync+inotify to synchronize the server in real-time, where inotify is used to monitor file system events, and rsync is a widely used synchronization algorithm with the advantage of operating only the different parts of the file, So its advantage is much more than using the hook file system for mirroring synchronization.

The more current version of the Synchronizer is Inotify-tools, and the other is the Google Open-source Project Openduckbill (dependent on inotify-tools), both of which are based on scripting languages, The design idea is also to use inotify and rsync commands. Compared to the above two projects, the advantages of this project are:

Sersync is written in C + + and filters temporary files and duplicate file operations generated by the Linux system file system (see appendix, this filter script is not implemented), so it saves run-time and network resources when it is synchronized with rsync. So faster.

Compared to the above two items, Sersync configuration is very simple, where the bin directory has basically statically compiled 2 of files, with the bin directory of the XML configuration file directly to use.

In addition this project compared to other script open source projects, using multithreading for synchronization, especially in the synchronization of large files, can ensure that more than one server real-time synchronization state.

This project has the error handling mechanism, through the failure queue to the wrong file resynchronization, if still fail, the synchronization failed to sync the file as long as the set time.

This project with crontab function, just open in the XML configuration file, you can according to your request, the overall synchronization of a period of time. No additional configuration of the crontab feature is required.

This project socket and HTTP plug-in extensions to meet your two development needs.

II. Basic Structure

As shown in the figure above, the thread group thread is the daemon that waits for the thread queue, and when there is data in the queue, the thread group Daemon wakes up one at a time, and when the queue has more inotify events, it is all awakened to work together. The purpose of this design is to be able to handle multiple inotify events simultaneously, and to reuse the concurrency capabilities of the server (kernel *2+2).

is called a thread group thread, because each thread at work, depending on the number of servers to establish child threads, child threads can ensure that all files with each server simultaneously, when the file to sync larger, so that each remote server can be synchronized to obtain the files.

The service thread has three functions, first, to process the files that failed the synchronization, to synchronize the files again, and to generate a rsync_fail_log.sh script to record the failed events for the files that failed to sync again. Execute the script once every 10 hours and empty the script at the same time. The third function of the service thread is the crontab function, which synchronizes all the paths once at a certain time.

Summarized by this diagram:

Sersync support multithreading, support queue filtering, save network bandwidth, failure retransmission mechanism, with SOCKET,HTTPD and other sockets, facilitate the development of two times.

Sersync also has the mechanism:

can achieve bidirectional synchronization, only need to be configured on two machines on the line;

During bidirectional synchronization, if a colleague modifies a file, the time is the

Latter

Third, Sersync installation and configuration specific steps

1.1.1.1 Server

1.1.1.2 Client

1, server-side installation

#wget http://sersync.googlecode.com/files/sersync2.5_32bit_binary_stable_final.tar.gz # tar XF sersync2.5_32bit_ Binary_stable_final.tar.gz # CD gnu-linux-x86/

Through the above simple three steps, you can complete its installation! After decompression, there are two files, a sersync2 and a confxml.xml, the first one is a binary file, that is, the file opening the service, the following is the configuration file!

The configuration file is explained below

<?xml version= "1.0"  encoding= "iso-8859-1"? >  #这里定义要监控的本地目录, this is important         <remote ip= "1.1.1.100"  name= "MP3"/>  #要同步到哪台服务器, What is rsync label         <!--<remote ip= "192.168.8.39"  name= " Tongbu "/>-->         <!--<remote ip=" 192.168.8.40 " name=" Tongbu "/>-->     </localpath>      <rsync>  #配置Rsync信息 &NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&NBSP;&LT;COMMONPARAMS&N bsp;params= "-artuz"/>  #rsync的参数         <auth start= " True " users=" Syncuser " passwordfile="/etc/rsyncd/rsyncd.pass "/>          <userdefinedport start= "false"  port= "874"/><!-- port=874  -->  # defines the rsync port         <timeout start= "false"  time = "/&gt";<!-- timeout=100 -->  #定义传输超时时间         <ssh  start= "false"/>  #Rsync的时候, whether to use SSH encryption     </rsync>    "/tmp/rsync_fail_log.sh"  timetoexecute= "  <faillog path="/><!--default  Every 60mins execute once-->     <crontab start= "false"  schedule= ><!--600mins-->  #定义crontab定期完全同步两端文件          <crontabfilter start= "false" >  #crontab同步时候的过滤条件, the top of the filter to open at the beginning of the          <exclude expression= "*.php" ></exclude>          <exclude expression= "info/*" ></exclude>          </crontabfilter>     </crontab>     &nbSp;<plugin start= "false"  name= "command"/>  the following are the settings for some plug-ins     </ Sersync>      <plugin name= "command" >      <param prefix= "/bin/sh"  suffix= " ignoreerror=" true "/>  <!--prefix  /opt/tongbu/mmm.sh suffix-->     <filter start= "false" >          <include expression= "(. *) \.php"/>          <include expression= "(. *) \.sh"/>      </filter>     </plugin>      < Plugin name= "Socket" >     <localpath watch= "/opt/tongbu" >          <deshost ip= "192.168.138.20"  port= "8009"/>      </locAlpath>     </plugin>     <plugin name= " Refreshcdn ">     <localpath watch="/data0/htdocs/cms.xoyo.com/site/">          <cdninfo domainname= "Ccms.chinacache.com"   port= " username=" "xxxx"  passwd= "xxxx"/>          <sendurl base= "Http://pic.xoyo.com/cms"/>         < Regexurl regex= "false"  match= "Cms.xoyo.com/site" ([/a-za-z0-9]*). Xoyo.com/images "/>  "    </localpath>     </plugin>  
Because rsync here, use the authentication user and password, so to define the password file
 
# Vim/etc/rsyncd/rsyncd.pass 123456
 
Then the permission is changed to 600.
 
# chmod 600/etc/rsyncd/rsyncd.pass
 
The above is a simple description of the configuration file, the following detailed say!