Rsync + sersync Real-time Data Synchronization directory

Chapter 2 rsync overview 1st basic rsync Overview

Rsync is an open-source backup tool that can be synchronized between different hosts to achieve full backup and Incremental backup.

Full: Transfer and overwrite all data

Incremental: Only data in the differential part is transmitted.

1.2 How incremental replication works

Rsync uses its unique "quick check" algorithm to incrementally transmit data

When synchronizing backup data, by default, rsync uses its unique "quick check" algorithm to synchronize only the files or directories whose size or last modification time has changed, of course, you can also synchronize data based on changes in permissions, owner, and other attributes, but you need to specify the relevant parameters. You can even synchronize only the changed content of a file, you can quickly synchronize backup data.

1.3 rsync software features

Similar to the CP command -- realize local backup and transmission of data similar to the SCP command -- remote backup and transmission of data similar to the RM command -- implement non-differential synchronous backup similar to the LS command -- local file information view rsync command is 1 V 4 command

1.4 remote copy

# Use rsync to copy the oldboyedu.com file under the/root directory on the 41 host and copy it to the local directory rsync-avz [email protected]:/root/oldboyedu.com. /# copy the file rsync-avz [email protected]:/root // backup/# Pull all files in the remote directory rsync-avz [email protected]: /root/backup/# Pull all files in the remote directory and Directory

Note: For remote directory copying, note that/

# Push the local/backup/2017-7-23 directory and content in the directory to rsync-avz/backup/2018-07-23 in the/tmp directory of the 41 server [email protected]:/tmp/

Problem:

1. the SSH system users used are insecure.

2. transmission is allowed only through the SSH protocol.

1.5 daemon copy

The daemon copy process is secure, and virtual users are used.

// Push local data to the backup server

rsync -avz /mnt/ [email protected]::backup/

// Pull the backup server data to the local server

rsync -avz [email protected]::backup/ /mnt/

Chapter 2 rsync server installation 2nd System Environment

[[email protected] ~]# cat /etc/redhat-release CentOS Linux release 7.1.1503 (Core) [[email protected] ~]# uname -r3.10.0-229.el7.x86_64[[email protected] ~]# systemctl status firewalldfirewalld.service - firewalld - dynamic firewall daemon   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled)   Active: inactive (dead)[[email protected] ~]# getenforceDisabled

2.2 install rsync2.2.1 check and install

$ Rpm-Qa | grep rsync # Check for installation, skip the following steps if installation does not, install rsync-3.1.2-4.el7.x86_64 $ Yum-y install rsync

2.2.2 Add a configuration file

$ Vim/etc/rsyncd. confuid = rsync # Not necessarily rsync, but WWW, etc, set the owner of the same directory as your site. The most suitable gid = rsyncport = 873 fake super = yesuse chroot = nomax connections = 200 timeout = 600 ignore errorsread only = falselist = falsehosts allow = 172.16.1.0/ 24 # modify the allowed CIDR blocks, enter hosts deny = 0.0.0.0/32 auth users = rsync_backupsecrets file =/etc/rsync as needed. passwordlog file =/var/log/rsyncd. log ##################################### [backu P] # Module name comment = welcome to oldboyedu backup! Path =/backup # Backup Directory

2.2.3 create a virtual user

$ useradd -M -s /sbin/nologin rsync

2.2.4 username and password for creating a virtual account (account and password used for client connection)

$ vim /etc/rsync.passwordrsync_backup:123456$ chmod 600 /etc/rsync.password

2.2.5 create a warehouse directory and authorize owner and group

$ mkdir /backup$ chown -R rsync.rsync /backup/

2.2.6 start

$ Systemctl start rsyncd # Start $ systemctl enable rsyncd # start automatically # Check whether the port has been started $ netstat-lntup | grep 873tcp 0 0 0.0.0.0: 873 0.0.0.0: * Listen 1861/rsync tcp6 0: 873: * Listen 1861/rsync

Chapter 2 client configuration 3rd client configuration to push or pull data without a password

$ echo "123456" > /etc/rsync.password$ chmod 600 /etc/rsync.password

Client test push File

$ Rsync-avz/etc/hosts [email protected]: backuppassword: # the password in the file is not the password of the Linux server, here is 123456 sending incremental file listsent 42 bytes received 20 bytes 17.71 Bytes/sectotal size is 158 speedup is 2.55 # This indicates that it has been passed, you can check whether this file exists in the server directory.

Chapter 4 install sersync for real-time monitoring and synchronization 4th install 4.1.1 sersync installed on the server you want to monitor, such as this directory changes frequently (client)

$ Mkdir/Server/tools-p $ CD/Server/tools $ wget https://raw.githubusercontent.com/wsgzao/sersync/master/sersync2.5.4_64bit_binary_stable_final.tar.gz # download sersync $ tar xf sersync2.5.4_64bit_binary_stable_final.tar.gz $ MV GNU-Linux-x86 // usr/local/sersync $ CD/usr/local /sersync/$ CP confxml. XML confxml. bak # X back up the configuration file before modification

4.1.2 Configuration

$ Vim confxml. xml 5 <filesystem XFS = "true"/> <! -- File System --> 6 <filter start = "false"> <! -- Exclude files that do not want to be synchronized --> 7 <exclude expression = "(. *)\. SVN "> </exclude> 8 <exclude expression = "(. *)\. GZ "> </exclude> 9 <exclude expression =" ^ info/* "> </exclude> 10 <exclude expression =" ^ static/* "> </exclude> 11 </filter> 12 <inotify> <! -- Monitored event type --> 13 <Delete start = "true"/> 14 <createfolder start = "true"/> 15 <createfile start = "true"/> 16 <closewrite start = "true"/> 17 <movefrom start = "true"/> 18 <moveTo start = "true"/> 19 <attrib start = "false"/> 20 <modify start = "false"/> 21 </inotify> 23 <sersync> 24 <localpath watch = "/var/www"> <! -- Monitored directory --> 25 <remote IP = "10.0.0.10" name = "backup"/> <! -- Backup IP address and module --> 26 <! -- <Remote IP = "192.168.8.39" name = "tongbu"/> --> 27 <! -- <Remote IP = "192.168.8.40" name = "tongbu"/> --> 28 </localpath> 29 <rsync> <! -- Rsync options --> 30 <commonparams Params = "-az"/> 31 <auth start = "true" users = "rsync_backup" passwordfile = "/etc/rsync. password "/> 32 <userdefinedport start =" false "Port =" 874 "/> <! -- Port = 874 --> 33 <timeout start = "true" time = "100"/> <! -- Timeout = 100 --> 34 <SSH start = "false"/> 35 </rsync> <! -- Execute synchronization every 60 minutes --> 36 <faillog Path = "/tmp/rsync_fail_log.sh" timetoexecute = "60"/> <! -- Default every 60 mins execute once -->

4.1.3 start

$. /Sersync2-hset the system paramexecute: Echo 50000000>/proc/sys/fs/inotify/max_user_watchesexecute: echo 327679>/proc/sys/fs/inotify/max_queued_eventsparse the Command Param ___________________________________________________ parameter-D: Enable daemon mode parameter-R: Before monitoring, push the monitoring directory and remote host again using the rsync command C parameter-N: specifies the number of daemon threads enabled. The default value is 10-O: Specifies the configuration file. By default, confxml is used. XML file parameter-M: other modules are enabled separately. Use-M refreshcdn to enable and refresh the CDN module parameter-M: enable other modules separately and use-M socket to enable Start socket module parameter-M: enable other modules separately. Enable the HTTP module with-m http without the-M parameter, the synchronization program $/usr/local/sersync/sersync2-dro/usr/local/sersync/confxml is executed by default. XML # Run the startup command to check whether the files in the monitoring directory are the same as those in the directory of another server.

　　

 

Rsync + sersync Real-time Data Synchronization directory