CentOS 7 rsync

CentOS 7 rsync

1) Software Introduction

Rsync is a remote data synchronization tool that enables fast synchronization of files between multiple hosts via Lan/wan. Rsync was originally a tool used to replace RCP, which is currently maintained by Rsync.samba.org. Rsync uses the so-called "rsync algorithm" to synchronize files between local and remote two hosts, which transmits only the different portions of two files, rather than the entire transfer every time, so it is quite fast. The machine running Rsync server is also called Backup server, a rsync server can back up data from multiple clients at the same time, or multiple rsync servers can back up one client's data.

Rsync can be paired with rsh or SSH or even use daemon mode. Rsync server will open a 873 service channel (port), waiting for the other rsync connection. When connected, Rsync server checks that the password matches and, if checked with a password, can start the file transfer. When the first connection is complete, the entire file is transmitted once, and the next time only the different portions of the two files are transmitted.

Rsync supports most Unix-like systems, which are well tested on Linux, Solaris, and BSD. In addition, it also has the corresponding version under the Windows platform, the more well-known have cwrsync and Sync2nas.

The basic features of Rsync are as follows:

Can be mirrored to save the entire directory tree and file system;
can easily do to maintain the original file permissions, time, soft and hard links, etc.;
can be installed without special permission;
Optimized process, high efficiency of file transfer;
You can use RCP, ssh and other means to transfer files, of course, you can also through a direct socket connection;
Supports anonymous transmissions.

2) Core algorithm

It is assumed that similar files A and B are synchronized between two computers named Alpha and beta, where α has access to file a, and beta has access to file B. and assume that the network bandwidth between the host α and beta is very small. Then the Rsync algorithm will be completed in the following five steps:

Beta splits file B into a set of data blocks that do not overlap a fixed size of S bytes. The last piece may be smaller than S.
Beta performs two checks on each partitioned block of data: One is a 32-bit scrolling weak check and the other is a 128-bit MD4 strong check.
Beta sends these check results to alpha.
Alpha searches for a block of data that has the same weak check code and strong check code as a piece of file B by searching for all data blocks of size S of file A (the offset can be optional, not necessarily a multiple of s). This work can be done quickly with the feature of the rolling check.
Alpha is sent to Beta for a sequence of instructions to generate a backup of file a on beta. Each instruction here is either a proof that a file B has a block of data that is not to be re-transmitted, or a block of data that must not match any of the data blocks of file B.

3) configuration Management

1 Turn off SELinux

Temporary shutdown Setenforce 0

Permanently close $vi/etc/sysconfig/selinux
Selinux=disabled

2 Installing the rsync system comes with

Boot Systemctl Enable RSYNCD

3 Configuring the service side
$ vi/etc/rsyncd.conf

#/etc/rsyncd:configuration File forrsync daemon mode# See rsyncd.conf Mans page forMore options.# configuration Example:uid=Rootgid=RootUse chroot=Nomax Connections=2Strict Modes=Yesport=873pid File= /var/run/Rsyncd.pidLockFILE =/var/run/rsync.Locklog File= /var/log/Rsyncd.log[hadoop]path=/opt/hadoop/etc/hadoop/# Data Storage location Comment=Hadoopread only=yeslist=Noauth Users=Hadoop # Unrelated to System users secrets file=/etc/rsyncd.secret#hosts Allow=192.168.6.1#设置允许访问的客户端多个客户端用空格隔开 #hosts deny= *#设置不允许访问的黑名单 [Zookeeper] #这里相当于一个区分符, if you want to synchronize more than one folder here, differentiate the path=/opt/zookeeper/conf # data storage location Comment=Zookeeperauth Users=hadoopsecrets File=/etc/Rsyncd.secrets # file that holds the rsync password read only=nolist= No

Configure the rsync password (the path is already written in the configuration file above)/etc/rsyncd.secrets (the name can be written as long as it is consistent with the top configuration file), format (one user at a line)
$ vi/etc/rsyncd.secrets

Hadoop:hadoop # User: Password Hadoop1:hadoop1

Configure rsync Password file permissions

Chown Root.root/etc/rsyncd.secrets
chmod 600/etc/rsyncd.secrets

4 Start verification

$ systemctl Start RSYNCD

$ NETSTAT-APN | grep rsync
TCP 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 2688/rsync
TCP6 0 0::: 873:::* LISTEN 2688/rsync

5 Client Configuration

1 Configure the three processes to do it.
1.1 Setting the password file
1.2 Testing rsync Execution Instructions
1.3 Putting the rsync instruction into the cycle task (crontab)

Set Password file
$ vi/etc/rsyncd.secrets
Hadoop #这里只需要写密码

$ chown Root:root/etc/rsyncd.secrets # Note must give permission
$ chmod 600/etc/rsyncd.secrets # must modify permissions

Test rsync Execution Instructions

Upload Rsync-avz--password-file=/etc/rsyncd.secrets/opt/hadoop/etc/hadoo [email protected]ode::hadoop
Download Rsync-avz--progress--password-file=/etc/rsyncd.secrets [email protected]::hadoop/opt/hadoop/etc/hadoop

Description of the parameters of rsync:
-A is equivalent to a collection of-rlptgod
-U is equivalent to –update and is not updated when the target file is newer than the source file
-V Show Synchronized files
–progress show the percent progress, transfer rate for file synchronization

Put the rsync command into the Cycle task (crontab)

CentOS 7 rsync