How to use the rsync file synchronization/transmission tool and incremental backup transmission tool in CentOS

Install rsync

CentOS/Fedora/RHEL: yum install rsync
Debian/Ubuntu: apt-get install rsync

Rsync tutorial

Rsync works in two ways: SSH and daemon.

SSH means that users on the system perform synchronization through SSH.
In daemon Mode, The rsync server opens port 873 and waits for the client to connect. The connection is verified and then synchronized.
1. Perform synchronization in SSH mode

Rsync-options SRC DEST
SRC indicates the source directory, and DEST indicates the target directory. When you connect to the directory through ssh, you must specify the ssh Port. Use this method: rsync-options-e "ssh-p 10000" SRC DEST

The following are examples of common scenarios:

1.1: Copy the/home/wwwroot/vpser/directory to/data/wwwroot/vpser/

Rsync-avu/home/wwwroot/vpser // data/wwwroot/vpser/

1.2: Copy the local directory to the remote server

Rsync-avu -- progress -- delete/home/wwwroot/vpser.net/root@1.2.3.4:/home/wwwroot/vpser.net/

If you change the SSH port, you need to add-e "ssh-p your SSH Port". If you do not want to display the specific transmission process, you can remove the P parameter. If you need to compress the transmission, you can add the z parameter.

-- Delete parameter. In this way, when a local file is deleted, the remote end will also delete the file to maintain the complete consistency.

1.3: Copy the remote server directory to the local device

Rsync-avu -- progress -- delete root@1.2.3.4:/home/wwwroot/vpser.net // home/wwwroot/vpser.net/

II. daemon mode configuration

The rsync daemon mode runs as an rsync server. First, we need to create a configuration file for the rsync server, which is/etc/rsyncd. conf by default, this file may not exist and needs to be created by yourself. The configuration information is as follows:

Port = 873
Uid = root
Gid = root
Use chroot = no // use chroot to file system directory
Max connections = 100 // maximum number of concurrent connections

# Motd file =/etc/rsyncd. motd // define the server information and write the rsyncd. motd file content by yourself
Pid file =/var/run/rsyncd. pid
Lock file =/var/run/rsyncd. lock
Log file =/var/log/rsyncd. log

Timeout = 300

[Vpser] // custom module
Path =/home/wwwroot/vpser.net // specifies the directory to be backed up
Ignore errors // Some IO errors can be ignored.
Read only = no // Set no. The client can upload files. yes is read-only.
Write only = no // no indicates that the client can be downloaded. yes indicates that the client cannot be downloaded.
Hosts allow = 192.168.2.0/24 // IP addresses that can be connected. We strongly recommend that you add IP addresses that you allow and separate them by commas.
List = yes // The list of modules used for client requests
Auth users = vpser // connection username, which is a virtual user that has nothing to do with linux users. Multiple usernames are separated by commas (,).
Secrets file =/etc/rsyncd. secrets // verify the password file. Format: user name: password

When writing the configuration, remove the preceding comments and adjust the relevant parameters. We also provide a template file http://soft.vpser.net/sync/rsync/rsyncd.conf to download and put it under/etc/and modify the relevant parameters.

The permission for the/etc/rsyncd. secrets file must be 600. After the file is created, run: chmod 600/etc/rsyncd. secrets.

Note: The default rsync server is not automatically started!

Debian/Ubuntu has a self-starting script. Modify/etc/default/rsync and change RSYNC_ENABLE = false to RSYNC_ENABLE = true to save the script and set it to boot automatically.

The boot script on CentOS is not included, run: wget http://soft.vpser.net/sync/rsync/init.d.rsync-O/etc/init. d/rsync & chmod + x/etc/init. d/rsync & chkconfig rsync on will start automatically if no error is reported.

After completing the preceding settings, run:/etc/init. d/rsync start to start the instance.

Test rsync server: rsync-avuP vpser@192.227.165.35: vpser/home/wwwroot/vpser.net/for connection testing, note: @ Before the vpser for the user name set in the custom module ,:: the following vpser defines the name of the module for you.

Tips for rsync:

1. You can use rsync to quickly delete a large number of files:
Create an empty folder: mkdir/root/blank
Use rsync to delete the target Directory: rsync -- delete-before-a-H-v -- progress -- stats/root/blank // home/wwwroot/cache/
In this way, the cache directory to be deleted will be cleared, and the deletion speed will be very fast.

2. If the iptables firewall is enabled, add Port 873 to the firewall's permit rules.
Iptables-I INPUT-p tcp -- dport 873-j ACCEPT
Iptables-I OUTPUT-p tcp -- sport 873-j ACCEPT

3. You can select the SSH or daemon mode as needed.
With crontab, tasks are scheduled to automatically complete synchronization and backup.

Common rsync parameters are described as follows:

-V, -- verbose detailed mode output
-Q, -- quiet simplified output mode
-C, -- checksum: enable the verification switch to force file transfer verification
-A, -- archive mode, indicating that the file is transmitted recursively and all file attributes are kept, equal to-rlptgoD.
-R, -- recursive processes subdirectories in recursive mode
-B, -- backup creates a backup, that is, if the object already has the same file name, rename the old file ~ Filename. You can use the -- suffix option to specify different backup file prefixes.
-Suffix = SUFFIX defines the backup file prefix.
-U, -- update only performs updates, that is, skipping all files that already exist in DST and whose file time is later than the time to be backed up. (Do not overwrite the updated file)
-L, -- links retains soft links
-P, -- perms to keep file permissions
-O, -- owner keeps file owner information
-G, -- group: Keep file group information
-T, -- times preserve the file time information
-E, -- rsh = COMMAND specifies that rsh and ssh are used for data synchronization.
-- Delete: delete the files that are not in the SRC file in DST.
-- Delete-excluded: delete files specified by this option at the receiving end.
-- Delete-after: delete after transmission
-- Ignore-errors is deleted when an IO error occurs in a timely manner.
-- Force directory deletion, even if not empty
-- Timeout = time ip timeout, in seconds
-- Progress displays the backup process
-Z, -- compress compresses backup files during transmission
-- Exclude = PATTERN specifies to exclude file modes that do not need to be transmitted
-- Include = PATTERN specifies the file mode to be transmitted without exclusion
-- Exclude-from = FILE: exclude files in the specified mode in the FILE.
-- Include-from = FILE: files with the specified FILE pattern matching are not excluded.