Rsync principles and deployment

Rsync Introduction

1. What is rsync?

Rsync is an excellent open-source, fast, and multi-function tool that can synchronize and back up incremental local or remote data images. Applicable to multiple platforms. The software name indicates remote sync ). Allows you to quickly synchronize images and remote backup of data between different partitions or directories on the local host and between local and remote hosts.

During backup synchronization, by default, rsync uses its unique "quick check" algorithm to synchronize only the files or directories whose size or last modification time has changed (you can also set the permission, parameters need to be set to synchronize changes such as owner ). You can even synchronize only the changed content of a file, so you can synchronize data quickly.

Tip: In the traditional cp, the scp tool copies a complete copy each time. In addition to the complete copy function, rsync also provides the incremental copy function, so the performance and efficiency are superior.

Rsync has the following features:

1) supports copying special files, such as links and devices.

2) You can exclude the synchronization of specified files or directories, which is equivalent to the packaging command tar.

3) You can keep the permissions of the original file or directory, and keep all attributes such as time, hard and soft links unchanged.

4) incremental synchronization is implemented, that is, only the changed data is synchronized, so the data transmission efficiency is higher.

5) You can use rcp, rsh, ssh, and other methods to transfer files or directly use socker links.

6) supports anonymous or authenticated process transmission to facilitate data backup and mirroring.

 

[1] Core algorithms:

Assume that similar files A and B are synchronized between two computers named α and beta. Alpha has access to file A and beta has access to file B. In addition, it is assumed that the network bandwidth between host α and β is very small. Then, The rsync algorithm is completed in the following five steps:

1. Beta splits file B into a group of data blocks with a fixed size of S bytes that do not overlap. The last part may be smaller than S.

2. Beta performs two types of verification on each split data block: one is a 32-bit rolling weak verification, and the other is a 128-bit MD4 strong verification.

3. Beta sends the verification results to α.

4. α searches all data blocks of file A with the size of S (the offset can be optional, not necessarily A multiple of S ), to find data blocks with the same weak verification code and strong verification code as one of file B. This work can be quickly completed using the features of rolling verification.

5. Alpha sends A string of instructions to generate A backup of file A on Beta. Each instruction here is either a proof that file B has a data block and does not need to be re-transmitted, or a data block, this data block certainly does not match any data block of file B.

------------------------------------- Summary ------------------------------------------------

Rsync synchronizes the data between directories at both ends. This data can be special data. Before synchronization, we will first compare the data at both ends, only synchronize different parts between the two, and retain the original attributes of the file. And supports anonymous synchronous transmission. Therefore, rsync is relatively fast during Backup.

Bytes -----------------------------------------------------------------------------------------

Ii. Software Installation

Http://rsync.samba.org/rsync Official Website

1) Compile and install

It's easy to decompress the package directly.

Tar zxf rsync-3.1.0.tar.gz

Cd rsync-3.1.0

./Configure

Make

Make install

Tip: You must install rsync on both the server and client.

Iii. server-side rsync deployment

Rsync startup

1. The Daemon starts rsync.
[Root @ www rsync-2.4.6] #/usr/local/bin/rsync -- daemon
To start. The default service port of rsync is 873. The server receives anonymous or authenticated requests from the customer on this port.
2. boot
2.1) inetd. conf
Edit/etc/services and add the row rsync 873/tcp. The service port of the specified rsync is 873. Add/etc/inetd. conf and rsync stream tcp nowait root/bin/rsync -- daemon

Note: For xinetd, the setting method is similar.
2.2) join rc. local
Add the startup command to rc. local:/usr/local/bin/rsync -- daemon

 

Rsync works in two ways:

1) transfer local files

2) data transmitted between the server and the client

 

Transfer local files

Specific rsync command parameters will be discussed later

[Root @ CT5_6-32-220-NFS01tmp] # rsync-avz/tmp/dir01/tmp/dir02/

Building file list... done

./

Test01

Sent 105 bytes encoded ed 48 bytes 306.00 bytes/sec

Total size is 0 speedup is 0.00

[Root @ CT5_6-32-220-NFS01 tmp] # tree

.

| -- Dir01

| '-- Test01

| -- Dir02

| '-- Test01

# You can see that the copied copy is successful. Note that/dir01/indicates all contents in the directory/dir01 indicates the directory itself

In this example, the contents in the directory are copied. Therefore, add/dir01/. If you want to copy the entire/dir01 directory to/dir02, write/dir01 ---> note that there is no/

[Root @ CT5_6-32-220-NFS01 tmp] # rsync-avz/tmp/dir01/tmp/dir02/# copy the directory itself

Building file list... done

Dir01/

Dir01/test01

Sent 111 bytes encoded ed 48 bytes 318.00 bytes/sec

Total size is 0 speedup is 0.00

[Root @ CT5_6-32-220-NFS01 tmp] # tree

| -- Dir01

| '-- Test01

| -- Dir02

| -- Dir01

| '-- Test01

| '-- Test01

Rsync details: click here
Rsync: click here

Recommended reading:

Monitor host files and directories using inotifywait

Using inotify + rsync for Linux File batch update

Inotify-tools + rsync real-time file synchronization installation and configuration

Complete rsync synchronization Configuration

1 2 3 4 Next Page