Sersync to implement data synchronization configuration tutorial

Sersync based on Boost1.43.0,inotify api,rsync command development, mainly for server synchronization, Web mirroring and other functions. Software Author: Zhou. Project official website; https://code.google.com/p/sersync/

Sersync Advantages:

1, Sersync is written in C + +, and the Linux system file system generated temporary files and duplicate file operations to filter, so in conjunction with rsync synchronization, save the running time consumption and network resources. So faster.

2, compared to Inotify-tools + rsync and Openduckbill two projects, Sersync configuration is very simple, where the bin directory has basically statically compiled 2 files, with the bin directory of the XML configuration file directly use.

3, the other sersync than other open source projects, using multithreading for synchronization, especially in the synchronization of large files, to ensure that multiple servers to maintain real-time synchronization status.

4, Sersync has an error handling mechanism, through the failure of the queue to resynchronize the file, if still failed, the synchronization failed to synchronize the files as long as set.

5, Sersync with crontab function, just open in the XML configuration file, you can according to your request, the whole synchronization of a period of time. No additional configuration of the crontab feature is required.

6, Sersync socket and HTTP plug-ins to expand, to meet the needs of two times development.

Visible Sersync + rsync is more efficient than Inotify-tools + rsync data synchronization. (The frequency and number of files updated in a certain range)

I. Planning:

Data synchronization Source: Nfs-s1 ip:172.16.1.10 directory:/data/images

Data Synchronization Purpose: NFS-S2 ip:172.16.1.20 directory:/data/images

Second, the configuration steps:

1, in the NFS-S2 configuration rsync daemon, and start. NFS-S1 Configure rsync client.

2, in the NFS-S1 configuration Sersync.

Third, NFS-S2 configuration rsync daemon: (Note the directory for the corresponding other programs read and write permissions)

[Root@nfs-s2 ~]# ifconfig eth0|awk-f ' [:]+ ' ' Nr==2{print $} '
172.16.1.20
[Root@nfs-s2 ~]# Uname-r
2.6.32-504.el6.x86_64
[Root@nfs-s2 ~]#

[root@nfs-s2 ~]# useradd-s/sbin/nologin-m www.
[Root@nfs-s2 ~]# Mkdir/data/images-p
[Root@nfs-s2 ~]# chown-r www.www/data/images/

Edit/etc/rsyncd.conf

[Root@nfs-s2 ~]# cat/etc/rsyncd.conf
Uid=www
Gid=www
Use Chroot=true
Read Only=false
List=false
Ignore Errors=true
Hosts ALLOW=172.16.1.0/24
Hosts deny=0.0.0.0/0
Max connections=1000
PID File=/var/run/rsyncd.pid
Lock File=/var/run/rsyncd.lock
Log File=/var/log/rsyncd.log
Auth Users=sersync
Secrets file=/etc/rsync.passwd
[Images]
Comment=images from NFS-S1
Path=/data/images
[Root@nfs-s2 ~]#

Edit rsync Daemon User password file:

[Root@nfs-s2 ~]# Echo ' sersync:123456 ' >/etc/rsync.passwd
[Root@nfs-s2 ~]# cat/etc/rsync.passwd
sersync:123456
[Root@nfs-s2 ~]# chmod 700/etc/rsync.passwd

Start rsync daemon:

[Root@nfs-s2 ~]# rsync--daemon

To create the rsync daemon password file in nfs-s1:

[Root@nfs-s1 ~]# ifconfig eth0|awk-f ' [:]+ ' ' Nr==2{print $} '
172.16.1.10
[Root@nfs-s1 ~]# Uname-r
2.6.32-504.el6.x86_64
[Root@nfs-s1 ~]#

[Root@nfs-s1 ~]# Echo ' 123456 ' >/etc/rsync.passwd
[Root@nfs-s1 ~]# chmod 700/etc/rsync.passwd
[Root@nfs-s1 ~]#

Test:

[Root@nfs-s1 ~]# echo ' Test rsync daemon from nfs-s1 ' >rsync.test
[Root@nfs-s1 ~]# rsync-avz rsync.test sersync@172.16.1.20::images--password-file/etc/rsync.passwd
Sending incremental file list
Rsync.test

Sent bytes received bytes 40.40 bytes/sec
Total size is speedup is 0.30
[Root@nfs-s1 ~]#

Rsync Daemon configuration succeeded.

[Root@nfs-s2 ~]# Echo ' rsync--daemon ' >>/etc/rc.local
[Root@nfs-s2 ~]# tail-1/etc/rc.local
Rsync--daemon
[Root@nfs-s2 ~]#

Iv. NFS-S1 Installation Configuration Sersync:(directory permissions according to the actual situation set)

[root@nfs-s1 ~]# useradd-s/sbin/nologin-m www.
[Root@nfs-s1 ~]# mkdir-p/data/images
[Root@nfs-s1 ~]# chown-r www.www/data/images/
[Root@nfs-s1 ~]#

Download Sersync:

[Root@nfs-s1 src]# wget-q http://www.111cn.net/repos/sersync2.5.4_64bit_binary_stable_final.tar.gz
[Root@nfs-s1 src]# ls
Sersync2.5.4_64bit_binary_stable_final.tar.gz
[Root@nfs-s1 src]# Tar XF sersync2.5.4_64bit_binary_stable_final.tar.gz
[Root@nfs-s1 src]# ls
Gnu-linux-x86 sersync2.5.4_64bit_binary_stable_final.tar.gz
[Root@nfs-s1 src]#

[Root@nfs-s1 src]# MV Gnu-linux-x86/usr/local/sersync
[Root@nfs-s1 src]# cd/usr/local/sersync/
[Root@nfs-s1 sersync]# ls
Confxml.xml SERSYNC2
[Root@nfs-s1 sersync]# mkdir {Bin,conf,log}
[Root@nfs-s1 sersync]# ls
Bin conf confxml.xml Log Sersync2
[ROOT@NFS-S1 sersync]# mv Confxml.xml conf
[Root@nfs-s1 sersync]# MV Sersync2 Bin/sersync
[Root@nfs-s1 sersync]# tree.
.
├──bin
│└──sersync
├──conf
│└──confxml.xml
└──log

3 Directories, 2 files
[Root@nfs-s1 sersync]#

Edit Confxml.xml: (Fine grained configuration based on actual requirements)

[Root@nfs-s1 sersync]# Cat Conf/confxml.xml

Remote name= "Images" ip= "172.16.1.20"/>

[Root@nfs-s1 sersync]#

Start Sersync:

[Root@nfs-s1 sersync]#/usr/local/sersync/bin/sersync-r-d-o/usr/local/sersync/conf/confxml.xml
Set the system param
Execute:echo 50000000 >/proc/sys/fs/inotify/max_user_watches
Execute:echo 327679 >/proc/sys/fs/inotify/max_queued_events
Parse the command param
Option:-R rsync All of the local files to the remote servers before the Sersync work
Option:-D run as a daemon
Option:-o config XML name:/usr/local/sersync/conf/confxml.xml
Daemon Thread Num:10
Parse XML config file
Host Ip:localhost host port:8008
Daemon Start,sersync run behind the console
Use rsync password-file:
User is Sersync
Passwordfile is/etc/rsync.passwd
Config XML parse success
Please set/etc/rsyncd.conf Max connections=0 manually
Sersync Working thread = 1 (primary thread) + 1 (fail retry thread) + (daemon sub threads)
Max Threads Numbers is:22 = (Thread pool nums) + (Sub threads)
Please according your CPUs, use-n param to adjust the CPU rate
------------------------------------------
Rsync the directory recursivly to the remote servers once
Working Please wait ...
Execute command:cd/data/images && rsync-artuz-r--delete./sersync@172.16.1.20::images--password-file=/etc/ RSYNC.PASSWD >/dev/null 2>&1
Run the Sersync:
Watch path is:/data/images
[Root@nfs-s1 sersync]#

Test:

[Root@nfs-s1 sersync]# Touch/data/images/{1..100}.png

[Root@nfs-s2 ~]# ls/data/images/
100.png 19.png 28.png 37.png 46.png 55.png 64.png 73.png 82.png 91.png
10.png 1.png 29.png 38.png 47.png 56.png 65.png 74.png 83.png 92.png
11.png 20.png 2.png 39.png 48.png 57.png 66.png 75.png 84.png 93.png
12.png 21.png 30.png 3.png 49.png 58.png 67.png 76.png 85.png 94.png
13.png 22.png 31.png 40.png 4.png 59.png 68.png 77.png 86.png 95.png
14.png 23.png 32.png 41.png 50.png 5.png 69.png 78.png 87.png 96.png
15.png 24.png 33.png 42.png 51.png 60.png 6.png 79.png 88.png 97.png
16.png 25.png 34.png 43.png 52.png 61.png 70.png 7.png 89.png 98.png
17.png 26.png 35.png 44.png 53.png 62.png 71.png 80.png 8.png 99.png
18.png 27.png 36.png 45.png 54.png 63.png 72.png 81.png 90.png 9.png
[Root@nfs-s2 ~]#

[Root@nfs-s1 sersync]# rm-f/data/images/*.png
[Root@nfs-s1 sersync]# Mkdir/data/images/{png,jpeg,jpg}
[Root@nfs-s1 sersync]#

[Root@nfs-s2 ~]# ls/data/images/
JPEG jpg png
[Root@nfs-s2 ~]#

Test successful

[Root@nfs-s1 sersync]# Echo '/usr/local/sersync/bin/sersync-d-o/usr/local/sersync/conf/confxml.xml ' >>/etc/ Rc.local
[Root@nfs-s1 sersync]# tail-1/etc/rc.local
/usr/local/sersync/bin/sersync-d-o/usr/local/sersync/conf/confxml.xml
[Root@nfs-s1 sersync]#