Linux Server rsync synchronization settings

This article will introduce you to the detailed guide on rsync synchronization settings for linux servers. I hope some methods will be helpful to you.

I. Introduction (skip this step if you don't want to see it)
After the EMOS server is migrated, the new server is set up and tested. Next, it is very important to migrate the mail data in the old mail system completely. You can choose to copy the data, but it is inevitable that some emails will be lost (emails received during this period), so rsync is taken into account.

Rsync is a remote data synchronization tool that allows you to quickly synchronize files between multiple hosts through the LAN/WAN. Rsync is a tool used to replace rcp, which is currently maintained by rsync.samba.org. Rsync uses the so-called "Rsync algorithm" to synchronize files between the local and remote hosts. This algorithm only transfers different parts of the two files, instead of transmitting the entire file each time, therefore, the speed is quite fast. The machine running Rsync server is also called backup server. One Rsync server can back up data of multiple clients at the same time, or multiple Rsync servers can back up data of one client.

Rsync can be used with rsh, ssh, or even daemon mode. Rsync server opens a 873 service channel (port) and waits for the peer Rsync to connect. During connection, the Rsync server checks whether the Password Matches. If the password is checked, file transmission can be started. When the first connection is complete, the entire file will be transferred once, and only different parts of the two files will be transferred at the next time.

Rsync supports most Unix-like systems, including Linux, Solaris, and BSD. In addition, it also has corresponding versions on the windows platform, including cwRsync and Sync2NAS.

The basic features of Rsync are as follows:
1. images can be used to save the entire directory tree and file system;

2. It is easy to maintain the permissions, time, and soft and hard links of the original file;

3. installation without special permissions;

4. optimized process, high file transmission efficiency;

5. You can use rcp, ssh, and other methods to transmit files. Of course, you can also use a direct socket connection;

6. Anonymous transmission is supported.

Core algorithms:
Assume that similar files A and B are synchronized between two computers named α and beta. Alpha has access to file A and beta has access to file B. In addition, it is assumed that the network bandwidth between host α and β is very small. Then, The rsync algorithm is completed in the following five steps:

1. Beta splits file B into a group of data blocks with a fixed size of S bytes that do not overlap. The last part may be smaller than S.

2. Beta performs two types of verification on each split data block: one is a 32-bit rolling weak verification, and the other is a 128-bit MD4 strong verification.

3. Beta sends the verification results to α.

4. α searches all data blocks of file A with the size of S (the offset can be optional, not necessarily A multiple of S ), to find data blocks with the same weak verification code and strong verification code as one of file B. This work can be quickly completed using the features of rolling verification.

5. Alpha sends A string of instructions to generate A backup of file A on Beta. Each instruction here is either a proof that file B has a data block and does not need to be re-transmitted, or a data block, this data block certainly does not match any data block of file B.

 

Command:

The Command Format of rsync can be as follows:

Rsync [OPTION]... SRC DEST

Rsync [OPTION]... SRC [USER @] HOST: DEST

Rsync [OPTION]... [USER @] HOST: SRC DEST

Rsync [OPTION]... [USER @] HOST: SRC DEST

Rsync [OPTION]... SRC [USER @] HOST: DEST

Rsync [OPTION]... rsync: // [USER @] HOST [: PORT]/SRC [DEST]

For the preceding six command formats, rsync has six different working modes:

1) copy a local file. This mode is enabled when the SRC and DES paths do not contain a single colon ":" separator.

2) use a remote shell program (such as rsh and ssh) to copy the content of the local machine to the remote machine. This mode is enabled when the DST path address contains a single colon ":" separator.

3) use a remote shell program (such as rsh and ssh) to copy the contents of the remote machine to the local machine. This mode is enabled when the SRC address path contains a single colon ":" separator.

4) copy files from the remote rsync server to the local machine. This mode is enabled when the SRC path information contains the ":" separator.

5) copy files from the local machine to the remote rsync server. This mode is enabled when the DST path information contains the ":" separator.

6) List of remote machine files. This is similar to rsync transmission, but you only need to omit the local machine information in the command.

1. Configure rsync to synchronize data on the server (rsync and configuration are installed in the rpm package)
[Root @ mail home] # rpm-qa | grep rsync # Check whether the rsync package is installed in the system

Rsync-2.6.8-3.1

[Root @ mail CentOS] # rpm-ivh rsync-2.6.8-3.1.i386.rpm # manually install if not installed

[Root @ test rsync-3.0.4] # vim/etc/xinetd. d/rsync

 

 

1. Configure rsync server

Vi/etc/xinetd. d/rsync

Change disable = yes to no

 

Service rsync

{

Disable = no

Socket_type = stream

Wait = no

User = root

Server =/usr/bin/rsync

Server_args = -- daemon

Log_on_failure + = USERID

}

 

2 configure rsync to start automatically

[Root @ test etc] # chkconfig rsync on

[Root @ test etc] # chkconfig rsync -- list

Rsync on

 

3. Configure rsyncd. conf

[Root @ test etc] # vim rsyncd. conf

 

Uid = root

Gid = root

Use chroot = no

Max connections = 4

Strict modes = yes

Port = 873

Pid file =/var/run/rsyncd. pid

Lock file =/var/run/rsync. lock

Log file =/var/log/rsyncd. log

 

[Backup]

Path =/home/domains/kidslandchina.com

Comment = MailDirectory

Auth users = mail

Uid = root

Gid = root

Secrets file =/home/rsync. ps

Read only = no

List = no

 

 

4. Make sure that the rsync port number in etc/services is correct.

[Root @ test etc] # vim/etc/services

Rsync 873/tcp # rsync

Rsync 873/udp # rsync

 

5. Configure the rsync password (the path has been written in the preceding configuration file)/home/rsync. ps (name can be written at will, as long as it is consistent with the above configuration file), format (one row per user)

[Root @ test etc] # vi/home/rsync. ps

Mail: abc123

 

6. Configure rsync Password File Permissions

[Root @ test home] # chown root. root rsync. ps

[Root @ test home] # chmod 400 rsync. ps

 

7 STARTUP configuration

[Root @ test home] #/etc/init. d/xinetd restart

Stopping xinetd: [OK]

Starting xinetd: [OK]

 

8. If xinetd does not exist, install it.

[Root @ test home] # yum-y install xinetd

 

 

Start rsync server

Two Methods for RSYNC server startup

9. Start the rsync server (independently started)

[Root @ test home] #/usr/bin/rsync -- daemon

 

10. Start the rsync server (xinetd superprocess started)

[Root @ test home] #/etc/init. d/xinetd reload

 

11 add rc. local

In various operating systems, rc files are stored in different locations and can be modified so that rsync-daemon can be loaded during system startup.

[Root @ test home] # vi/etc/rc. local

/Usr/local/rsync-daemon # Add a row

 

12. Check whether rsync is enabled.

[Root @ test home] # lsof-I: 873

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME

Xinetd 4396 root 5u IPv4 633387 TCP *: rsync (LISTEN)

 

 

 

 

2. Configure the rsync client to synchronize data (rsync and configuration are installed in the rpm package)
 

1. Configure three processes.

1.1 SET Password File

1.2 test the rsync execution command

1.3 put the rsync command into the Job Schedule (crontab)

[Root @ aj1 home] # vi/etc/xinetd. d/rsync

 

# Default: off

# Description: The rsync server is a good addition to an ftp server, as it

# Allows crc checksumming etc.

Service rsync

{

Disable = yes

Socket_type = stream

Wait = no

User = root

Server =/usr/bin/rsync

Server_args = -- daemon

Log_on_failure + = USERID

}

 

1.1 configure the password file (NOTE: For security purposes, set the attribute of the password file to 600. The rsync. ps password must be the same as the password in the Rsync Server password setting case)

[Root @ aj1 home] # vi rsync. ps

Abc123

 

[Root @ mail home] # chown root. root. rsync. ps # note that you must grant permissions

[Root @ mail home] # chmod 600. rsync. ps # The permission must be modified.

 

1.2 download files from the server

Rsync-avz -- password-file =/home/rsync. ps mail@10.100.0.175: backup/home/domains/kidslandchina.com/

[Root @ mail rsync-3.0.4] # rsync-avz -- password-file =/home/rsync. ps scihoo@192.168.0.206: backup/home/

 

Upload from local to server

[Root @ mail rsync-3.0.4] # rsync-avz -- password-file =/home/rsync. ps/home scihoo@192.168.0.206: backup

 

 

 

 

Troubleshooting:
1. Question 1: rsync: failed to connect to X. X: No route to host (113)
The backup task sent by rsync is not successfully executed. As follows:

Rsync: failed to connect to X. X: No route to host (113)

Rsync error: error in socket IO (code 10) at client server. c (107) [sender = 2.6.8]

 

The two hosts can ping each other, but the error message says they have not been routed to the host!

Fault cause: the other party is not started, the firewall is blocked, and the firewall is blocked on the network.

Solution: Disable the firewall or open tcp udp port 873 of the firewall to allow rsync to pass.

Implementation Scheme:

1. Firewall Start and Stop

# Service iptables start/stop

2. Allow rsync to pass through the firewall

To prevent the rule from being cleared after a restart, I added the rule to the rule configuration file (/etc/sysconfig/iptables) as follows:

-A input-p tcp-s X. X -- dport 873-j ACCEPT

Note: This rule must be added before the REJECT rule.

Restart iptables (# service iptables restart ).

 

 

2. Question 2: @ ERROR: auth failed on module backup

@ ERROR: auth failed on module backup

Rsync error: error starting client-server protocol (code 5) at main. c (1522) [runner ER = 3.0.3]

Cause:

The backup module on the server needs to verify the user name and password, but the client does not provide the correct user name and password. Authentication fails. Provide the correct username and password to solve this problem.

Check the password of the/home/rsync. ps file specified by the backup module in/etc/rsyncd. conf on the server.