The rsync thing.

The rsync thing. 1. About Rsync

Rsync is the short name of remote synchronize, is a long-range synchronization software, in the project more useful. The rsync algorithm used (the core algorithm for the rsync core algorithm on the cool shell) provides a faster way for client and remote file server synchronization.

Rsync is characterized by the ease of installation and configuration, and the ability to synchronize files while maintaining symbolic links, hard links, permissions, file time, document attribution, and more. Security provides password, read-write permission control, and IP rights control.

2.rsync Installation and Configuration

Since I'm using Ubuntu, I've installed rsync, so I don't have to install it. My configuration file is/etc/rsyncd.conf, which is configured as follows, see Rsyncd.config for more detailed configuration:

read only = yeslistfile = /var/runfile = /etc/rsyncd.secretslogfile = /var/log/rsync.log127.0.0.110.211.55.11auth users = rsynctest

Where read only sets the synchronization directory to read-only, UID and GID are used by users and user groups for synchronization. The PID file and log file store addresses for PID files and journal files.

Test is the synchronization directory, path is the synchronization directory, note that the variable is annotated, such as #xxx, plus the reported chroot error. In addition, hosts allow is limited to synchronous IP, with multiple IP spaces separated. Auth Users and Secrets file are authenticated user and password files, respectively. Where the secrets file permission must be 600, otherwise the synchronization will be error, the format of the user name: password, here the user name is not necessarily a system user, can be arbitrarily named, such as mine is rsynctest:testpasswd .

3.rsync Sync Command

There are two modes of rsync synchronization, a connection via a remote shell such as SSH, and a direct connection to the Rsync server via TCP. Because the use of SSH and other synchronization needs to set up the SSH corresponding port, the more troublesome point, I generally use the direct connection to the Rsync server this way. The most common way to synchronize commands is as follows:

rsync  - avz  - - progress  - - delete  rsync://[email  Protected]   211   55   11/test/ rsynctarget/  

Where rsynctest is my sync username, 10.211.55.11 is the IP of my rsync server. Run the command and enter the password. If you do not want to enter the password every time, you can add parameters --password-file=/etc/rsyncd.secrets , wherein Rsyncd.secrets is your rsync client machine files, permissions to 600, content format and rsync server is not the same, here only need password testpasswd can. –progress shows the progress of the synchronization, –delete is to delete the extra files in the Rsynctarget.

These options AvZ is very common, where V is verbose,z is compression, a equivalent to Rlptgod, where R is a recursive synchronization directory, if not add-r, but with-D, will only synchronize the corresponding directory, but not the directory under the file; L synchronize symbolic Links, p sync files keep permissions unchanged, T keep the file time, g keep the file belongs to the group, O keep the file belongs to the user, D keep the device files. The general situation with AVZ can be. There is also an option-P will also be used, indicating that the file is kept partially synchronized, if the synchronization is interrupted, can be continued.

There is another place to note, is in the synchronization of a subdirectory, such as I sync rsynctest/testdir this directory, if the command is this rsync -avz rsync://[email protected]/test/testdir rsynctarget/ , because TestDir does not add "/", The result of the synchronization is that there will be a TestDir directory under the Rsynctarget directory, that is, Rsynctarget/testdir. If the command is `rsync -avz rsync://[email protected]/test/testdir/ rsynctarget/ , the result is to synchronize all the files under the TestDir directory to the Rsynctarget directory, Rsynctarget directory will not have testdir this directory.

4. Example

For convenience, I built a directory to do the rsync test. The contents of the directory Rsynctest are as follows:

Root@ubuntu:/home/ssj/rsynctest# Ls-laTotal -Drwxr-xr-x3Nobody Nogroup4096 Mar  7  -: -. drwxr-xr-x -SSJ SSJ4096 Mar  7  -: -.. -rw-r--r--2Nobody Nogroup - Mar  7  -: +Link.txtlrwxrwxrwx1Nobody Nogroup A Mar  7  the: -Outer.txt. /outer.txtlrwxrwxrwx1Nobody Nogroup8 Mar  7  -: theSoftlink.txt-Test.txtdrwxr-xr-x2Nobody Nogroup4096 Mar  7  -:Wuyitestdir-rw-r--r--2Nobody Nogroup - Mar  7  -: +Test.txt

Where Softlink.txt is a symbolic link to test.txt, Link.txt is a hard link to test.txt, and Outer.txt is a hard link to the outer.txt of the previous level of the directory.

The contents of the synchronization target directory Rsynctarget are as follows:

[Email protected]-MBPSsj#ls -laRsynctarget/Total8Drwxr-XR- x   3 65534  65534   102  3  7  -: the .Drwxr-XR- x@94SSJ Staff3196  3  6  at: - ..-RW- R--R--   1Root65534     7  3  7  -: theMyFile.Txt

With different parameters, the results of the synchronization will not be the same.

1) Do not use the-a parameter, do not-OG, etc., the command is as follows:

rsync  - rpvz  - - progress  - - Span class= "hljs-comment" >delete rsync://[email protected]   211   55   11/test/ rsynctarget/ - - password  - FILE=/ETC/RSYNCD   secrets  

Due to the use of –delete, the original rsynctarget in the myfile.txt will be deleted, because there is no add-l, the symbolic link file outer.txt and softlink.txt is not synchronized. After synchronization, the hard link test.txt is treated as a new file, the hard link relationship is not preserved, and the-H option is required if you want to keep it. In addition, the file owner is not the original nobody:nogroup, if you need to keep, add-og parameters. After the command is synchronized, the contents of the Rsynctarget directory are as follows:

Root@ubuntu:/HOME/SSJ# Ls-la rsynctarget/Total -Drwxr-xr-x3Root root4096 Mar  7  -: to. drwxr-xr-x inSSJ SSJ4096 Mar  7  -: to.. -rw-r--r--1Root root - Mar  7  -: toLink.txtdrwxr-xr-x2Root root4096 Mar  7  -: totestdir-rw-r--r--1Root root - Mar  7  -: toTest.txt

2) Use the-a parameter. The command is as follows:

rsync  - avz  - - progress  - - Span class= "hljs-comment" >delete rsync://[email protected]   211   55   11/test/ rsynctarget/ - - password  - FILE=/ETC/RSYNCD   secrets  

After synchronizing the contents of the Rsynctarget directory, you can see that after using-a, the file owner is retained, the soft connection is also synchronized, not strong link relationship is not retained. To keep, add-H.

[Email PROTECTED]:/HOME/SSJ# Ls-la rsynctarget/Total -drwxr-xr-x  3Nobody Nogroup4096Mar7  -: -. drwxr-xr-x  inSSJ SSJ4096Mar7  -: to.. -rw-r--r--1Nobody Nogroup -Mar7  -: +Link. txtlrwxrwxrwx1Nobody Nogroup AMar7  the: -Outer. txt.. /outer. txtlrwxrwxrwx1Nobody Nogroup8Mar7  -: theSoftlink. txt-Test. txtdrwxr-xr-x  2Nobody Nogroup4096Mar7  -:Wuyitestdir-rw-r--r--1Nobody Nogroup -Mar7  -: +Test. txt

3) Another important parameter is-p, note that this P is not the meaning of progress, he is the meaning of partial, that is, the synchronization process if there is an interruption (such as active stop synchronization or network problems caused by synchronization interrupt), if a file is not finished, Whether to keep these transferred parts is controlled by-P. Here I add a test.tar.gz file under the Rsynctest directory, the size of 2M, and I use the Bwlimit parameter to limit the transfer speed of 10k/s, so that the midway termination of synchronization, so you can see, if there is no add-p parameter, Rsynctarget not save tornado.tar.gz This file has been transferred part of, if add-P will save the transferred part, the file name is also tornado.tar.gz, the next retransmission will continue to pass the breakpoint, that is, the synchronized parts will not be retransmitted. The temporary file name during synchronization is a format such as. Tornado.tar.gz.qesuQR. If a program is fetching data from the synchronization directory and processing it, it is best not to use the-p parameter, because if the network problem causes the file to be sent to a part and then processed, this will affect the processing result.

rsync -Avzp --Progress --Delete --bwlimit=10 Rsync://[email protected].211. -.11/test/ rsynctarget/ --Password-FILE=/ETC/RSYNCD.Secrets

5. References

• Rsync Security Configuration
• rsync Core algorithm
• Rsync remotely synchronizes backup data

The rsync thing.