Installation and configuration of rsync server in CentOS 6.3

Installation and configuration of rsync server in CentOS 6.3

I. Introduction to rsync

Remote synchronize (Rsync) is a remote data synchronization tool that allows you to quickly synchronize files between multiple hosts through the LAN/WAN, or use Rsync to synchronize different directories on the local hard disk.

Rsync is a tool used to replace rcp. Rsync uses the so-called "Rsync algorithm" to synchronize files between local and remote hosts. This algorithm only transfers different parts of two files, instead of sending the whole copy each time, the speed is quite fast. You can refer to How Rsync Works A Practical Overview to learn more about the operating mechanism of rsync.

Rsync supports most Unix-like systems, including Linux, Solaris, and BSD. In addition, it also has corresponding versions on the windows platform, including cwRsync and Sync2NAS.

The initial authors of Rsync are Andrew Tridgell and Paul macerras, which are currently maintained by the http://rsync.samba.org.

The basic features of Rsync are as follows:

1. images can be used to save the entire directory tree and file system;

2. It is easy to maintain the permissions, time, and soft and hard links of the original file;

3. installation without special permissions;

4. optimized process, high file transmission efficiency;

5. You can use rcp, ssh, and other methods to transmit files. Of course, you can also use a direct socket connection;

6. Anonymous transmission is supported to facilitate website images.

When using rsync for remote synchronization, two methods can be used: Remote Shell (ssh is recommended, and ssh is responsible for user authentication) and C/S (that is, the client connects to the remote rsync server, and the rsync server is responsible for user authentication ).

Regardless of the local synchronization directory or remote data synchronization, all the files will be copied once during the first run, and only the changed files will be copied later (for new files) or the change part of the file (for the original file ).

Rsync has no speed advantage in the first copy and is not as fast as tar. Therefore, when the data volume is large, you can use tar for the first copy before using rsync for data synchronization.

Ii. System Environment

System Platform: CentOS release 6.3 (Final)

Rsync version: rsync-3.0.9-2.el6.rfx.x86_64.rpm

Rsync server: TS-DEV (172.16.1.135)

TS-CLIENT (172.16.1.136)

Iii. Installing rsync on the server

3.1. Check whether rsync has been installed

# Rpm-qa | grep rsync

If it has been installed, run the rpm-e command to uninstall it.

3.2. Download the RPM package

# Wget http://pkgs.repoforge.org/rsync/rsync-3.0.9-2.el6.rfx.x86_64.rpm

3.3. Install rsync

# Rpm-ivh rsync-3.0.9-2.el6.rfx.x86_64.rpm

4. Configure the rsync Service

4.1. Procedure for configuring the rsync server

• First, select the server startup mode.
  • For rsync servers with heavy load, use an independent running mode.
  • Xinetd can be used for rsync servers with less load.
• Create the configuration file rsyncd. conf
• Create an authentication password file for non-Anonymous rsync servers

4.2. Run the rsync service with xinetd

By default, CentOS runs the rsync service in xinetd mode. Xinetd configuration file of rsync
In/etc/xinetd. d/rsync. To configure the rsync service running with xinetd, run the following command:

# chkconfig rsync on# service xinetd restart

The administrator can modify the/etc/xinetd. d/rsync configuration file to suit your needs. For example, you can modify the configuration line

server_args = --daemon

Add the rsync service options later.

4.3. Run the rsync service independently

The simplest way to run the rsync service independently is to execute the following command:

# /usr/bin/rsync --daemon

You can write the above command into the/etc/rc. local file to run the rsync service every time you start the server. Of course, you can also write a script to automatically start the rysnc service at startup.

4.4. configuration file rsyncd. conf

You must configure rsyncd. conf for both rsync service running modes. The format is similar to the main configuration file of samba.
The configuration file rsyncd. conf is in the/etc directory by default. To put all files related to the rsync service in a separate directory, run the following command:

# mkdir /etc/rsyncd# touch /etc/rsyncd/rsyncd.conf# ln -s /etc/rsyncd/rsyncd.conf /etc/rsyncd.conf

The configuration file rsyncd. conf consists of global configuration and several module configurations. The syntax of the configuration file is:

• The module starts with a [Module name ].
• The format of the parameter configuration line is name = value. value can be of two data types:
  • String (you can define a string without quotation marks)
  • Boolean value (1/0, yes/no, true/false)
• Comments starting with # or;
• \ Is the line feed.

Global Parameters

All configuration lines other than [module] in the file are global parameters. You can also define the module parameters in the global parameters section. The value of this parameter is the default value of all modules.

Parameters	Description	Default Value
Address	The IP address used for the specified server to run independently. This parameter is ignored when xinetd is running and replaced by the-address option on the command line.	All local IP addresses
Port	Specifies the port number listened by the rsync daemon. This parameter is ignored when xinetd is running and replaced by the-port option on the command line.	873
Motd file	Specify a message file. When the client connects to the server, the file content is displayed to the client.	None
Pid file	The rsync daemon writes its PID to the specified file.	None
Log file	Specify the log file of the rsync daemon, instead of sending the log to syslog.	None
Syslog facility	Specifies the message level when rsync sends a log message to syslog.	Daemon
Socket options	Specify custom TCP options.	None

Module Parameters

Module parameters are mainly used to define which directory of the rsync server to be synchronized. The format of the module Declaration must be in the [module] format. This name is the name seen on the rsync client, similar to the sharing name provided by the Samba server. The data actually synchronized by the server is specified through path. You can specify multiple modules as needed. The following parameters can be defined in the module:

A. Basic module Parameters

Parameters	Description	Default Value
Path	Specifies the synchronization path of the current module on the rsync server. This parameter must be specified.	None
Comment	Specify a description for the module. This description, together with the module name, is displayed to the customer when the customer connects to the module list.	None

B. module control parameters

Parameters	Description	Default Value
Use chroot	If it is true, rsync first chroot to the directory specified by the path parameter before transferring the file. The reason for this is to implement additional security protection, but the disadvantage is that the root permission is required, and the directory file to which the symbolic connection pointing to the external path cannot be backed up.	True
Uid	This module transfers files with the specified UID.	Nobody
Gid	Specify this module to transmit files with the specified GID.	Nobody
Max connections	Specify the maximum number of concurrent connections for this module to protect the server. connection requests that exceed the limit will be notified and then try again.	0 (no restrictions)
Lock file	Specifies the lock file that supports the max connections parameter.	/Var/run/rsyncd. lock
List	Specify whether the module should be listed when the customer requests to list available modules. If this option is set to false, you can create a hidden module.	True
Read only	Specifies whether the client is allowed to upload files. If the value is true, upload is not allowed. If the value is false and the server directory has read and write permissions, upload is allowed.	True
Write only	Specifies whether to allow users to download files. If the value is true, downloading is not allowed. If the value is false and the server directory has the read permission, downloading is allowed.	False
Ignore errors	Specifies whether to ignore the I/O error when running the delete operation on the rsync server. In general, rsync will skip the-delete operation when an I/O error occurs to prevent serious problems caused by temporary resources or other I/O errors.	True
Ignore nonreadable	Specify that the rysnc server ignores all files that the user does not have access. This makes sense when some files in the directories to be backed up should not be obtained by the backup owner.	False
Timeout	This option overwrites the specified IP timeout time. This ensures that the rsync server will not always wait for a crashed client. For anonymous rsync servers, the ideal number is 600 (in seconds ).	0 (unlimited)
Dont compress	Used to specify files that are not compressed before transmission. This option defines a list of Command Options that cannot be used by customers for this module. You must use the full name of the option, not the abbreviation. When an option is rejected, the server reports an error message and exits. For example, to prevent compression, it should be: "dont compress = *".	*. Gz *. tgz *. zip *. z *. rpm *. deb *. iso *. bz2 *. tbz

C. Module file filtering parameters

Parameters	Description	Default Value
Exclude	Specify multiple files or directories (relative paths) separated by spaces and add them to the exclude list. This is equivalent to using-exclude in client commands to specify the mode.	Null
Exclude from	Specifies a file name that contains the exclude rule definition. The server reads the exclude list definition from the file.	Null
Include	Specify multiple files or directories (relative paths) separated by spaces and add them to the include list. This is equivalent to using-include to specify the mode in client commands.	Null
Include from	Specify a file name that contains the include rule definition. The server reads the include list definition from the file.	Null

• One module can only specify one exclude parameter and one include parameter.
• Combined with include and exclude, you can define complex exclude/include rules.
• These parameters are equivalent to the corresponding rsync Client Command Options. The only difference is that they act on the server.
• For details about how to write the rule file, see http://www.howtocn.org/rsync:use_rsync.

D. Module User Authentication Parameters

Parameters	Description	Default Value
Auth users	Specifies a list of usernames separated by spaces or commas. Only these users can connect to this module. The user here has nothing to do with the System user. The username and password are stored in plaintext in the file specified by the secrets file parameter.	(Anonymous)
Secrets file	Specifies an rsync authentication password file. This file takes effect only when auth users is defined.	Null
Strict modes	Specifies whether to monitor the permissions of the password file. If this parameter is set to true, the password file can only be accessed by users with the running identity of the rsync server. Other users cannot access the file.	True

• The permission of the rsync authentication password file must be 600; otherwise, the client cannot connect to the server.
• In the rsync authentication password file, specify a username: Password pair in the format:

Username: passwd

• Generally, the password should not exceed 8 characters. If you only configure the rsync server for anonymous access, you do not need to set the preceding parameters.

E. Module Access Control Parameters

Parameters	Description	Default Value
Hosts allow	Use a host list to specify which host customers are allowed to connect to the module. The host that does not match the host list will be rejected.	*
Hosts deny	Use a host list to specify which host customers are not allowed to connect to this module.	Null

The client host list can be defined in the following format:

• A single IP address. Example: 192.168.0.1
• The entire network segment. Example: 192.168.0.0/24,192.168 .0.0/255.255.255.0
• A single host name that can be parsed. Example: centos, centos.bsmart.cn
• All hosts in the domain. Example: * .bsmart.cn
• "*" Indicates all.
• Multiple list items must be separated by spaces.

F. Module Log Parameters

Parameters	Description	Default Value
Transfer logging	Enables the rsync server to transfer operation records to the transfer log file.	False
Log format	Specifies the fields for transferring log files.	"% O % h [% a] % m (% u) % f % l"

When the "log file" parameter is set, "% t [% p]" will be added at the beginning of each log line.

You can use the following log format specifiers:

• % A-remote IP Address

• % H-remote host name

• % L-file length characters

• % P-PID of the rsync session

• % O-operation type: "send" or "recv"

• % F-file name

• % P-module path

• % M-Module name

• % T-current time

• % U-Authenticated User Name (null when anonymous)

• % B-Number of actually transmitted bytes

• % C-when a file is sent, the file verification code is recorded.

V. rsync server application cases

5.1. Configure the rsync service on the server-side TS-DEV

A. Edit the configuration file

# Vi/etc/rsyncd. conf

# Minimal configuration file for rsync daemon# See rsync(1) and rsyncd.conf(5) man pages for help# This line is required by the /etc/init.d/rsyncd script
# GLOBAL OPTIONSuid = root                         gid = root                                  
use chroot = no                             
read only = yes                            #limit access to private LANshosts allow=172.16.0.0/255.255.0.0 192.168.1.0/255.255.255.0 10.0.1.0/255.255.255.0hosts deny=*                                

max connections = 5                      

pid file = /var/run/rsyncd.pid             

secrets file = /etc/rsyncd/rsyncd.secrets   

#lock file = /var/run/rsync.lock           
motd file = /etc/rsyncd/rsyncd.motd        #This will give you a separate log filelog file = /var/log/rsync.log               #This will log every file transferred - up to 85,000+ per user, per synctransfer logging = yes                    log format = %t %a %m %f %bsyslog facility = local3timeout = 300
# MODULE OPTIONS[davidhome]                               path = /home/david/                       list=yes                                   ignore errors                              auth users = david                          comment = David home                  exclude = important/ [chinatmp]path = /tmp/china/list=noignore errorsauth users = chinacomment = tmp_china

B. Create the/etc/rsyncd. secrets file.

# Vim/etc/rsyncd. secrets

David: asdf # format Username: Password
China: jk # This user is not a system user

C. For Password Security, we set the permission to 600.

# Chown root: root/etc/rsyncd. secrets

# Chmod 600/etc/rsyncd. secrets

D. Create the welcome information file/etc/rsyncd. motd displayed on the Client Connected to the server.

# Vim/etc/rsyncd. motd

++
+ David Camp +
++

E. Start rsync

#/Etc/init. d/xinetd restart

F. Check whether port 873 is enabled.

# Netstat-an | grep 873

If rsync is successfully started, you can see that port 873 is already listening.

G. server-side file details

5.2. Client Configuration

A. Install rsync on the client

# Yum-y install rsync

B. Use the rsync client to synchronize data

Scenario 1:

# Rsync-avzP david@172.16.1.135: david home/tmp/david/

Password: Enter the david Password provided by the server. In the previous example, we used asdf, And the entered Password was not displayed. Press enter when the Password is set;
Note: This command means to log on to the server with the user david and synchronize the data of david home to the local directory/tmp/david. Of course, the local directory can be defined by yourself. For example, dave can also. When you do not have the "David home" directory on the client, the system will automatically create one for you. If the directory David home exists, pay attention to its write permission.

Note:
-A parameter is equivalent to-rlptgoD.-r indicates recursive-l indicates a link file, which means copying a link file;-p indicates retaining the original permissions of the file;-t indicates retaining the original time of the file; -g indicates the original user group of the file,-o indicates the original owner of the file, and-D indicates a block device file;
-Z compression during transmission;
-P transmission progress;
-V transmission progress and other information. It has something to do with-P. Try it on your own. You can see the document;

Scenario 2:
# Rsync-avzP -- delete david@172.16.1.135: david home/tmp/david/

This time, we introduce the-delete option, indicating that the data on the client must be exactly the same as that on the server. If the/tmp/david/directory contains files that do not exist on the server, delete them. The ultimate goal is to make the data in the/tmp/david/directory completely consistent with that on the server; Be careful when using it; it is best not to put the directory with important data, as a local update directory, otherwise all your data will be deleted;

Scenario 3:
# Rsync-avzP -- delete -- password-file =/tmp/rsync. password david@172.16.1.135: david home/tmp/david/

This time, we added the option-password-file = rsync. password. When we log on to the rsync server as david to synchronize data, the password will read the/tmp/rsync. password file. This file contains only the password of david. We need to do the following;

# Touch/tmp/rsync. password
# Chmod 600/tmp/rsync. password
# Echo "asdf">/tmp/rsync. password
# Rsync-avzP -- delete -- password-file =/tmp/rsync. password david@172.16.1.135: david home/tmp/david/

Note: In this case, no password is required. In fact, this is important because it is necessary for the server to schedule tasks through crond;

5.3. The rsync client automatically synchronizes data with the server

Edit crontab
# Crontab-e
Add the following code:

10 0 * * * rsync -avzP  --delete  --password-file=/tmp/rsync.password  david@172.16.1.135::davidhome  /tmp/david/

Indicates that the subsequent commands are executed at 00:10 every day.

Vi. Error Analysis

@ ERROR: chdir failed rsync error: error starting client-server protocol (code 5) at main. c (1530) [Cycler = 3.0.6]

Rsync: opendir "." (in xxxxxxx) failed: Permission denied (13)

Solution:

1. Remove all restrictions on rsync from selinux:
#/Usr/sbin/setsebool-P rsync_disable_trans 1
# Service xinetd restart

2. Click it to deny the whole selinux:
# Vim/etc/selinux/config
Change SELINUX = enforcing to SELINUX = disabled.
After saving and exiting, restart the machine.

Now, rsync server configuration is complete.

RSync for file backup Synchronization

Monitor host files and directories using inotifywait

Using inotify + rsync for Linux File batch update

Inotify-tools + rsync real-time file synchronization installation and configuration

Complete rsync synchronization Configuration

Remote synchronization of Rsync in CentOS 6.5

Rsync details: click here
Rsync: click here