A Concise handbook of rsync

A Concise handbook of rsync

!rsync Sync mode
Sync or backup, use remote shell, or TCP connection remote daemon, there are two ways to connect to the remote host.
Shell mode, no configuration file required, and no need to start the remote rsync. SSH is generally used as a transport tool when transmitting remotely.
Daemon mode, but the rsync must be started on a single machine.

!rsync Command Call format
Local file synchronization:
rsync [OPTION ...] Src... [DEST]
Example:
Rsync-a/home/back1/home/back2

Based on remote shell synchronization:
Pull: rsync [OPTION ...] [[email protected]] Host:src ... [DEST]
Push: Rsync [OPTION ...] Src... [[email protected]] Host:dest

Synchronization based on Rsync daemon:
Pull: rsync [OPTION ...] [[email protected]] Host::src ... [DEST]
rsync [OPTION ...] rsync://[[email protected]]host[:P ort]/src ... [DEST]
Push: Rsync [OPTION ...] Src... [[email protected]] HOST::D EST
rsync [OPTION ...] SRC ... rsync://[[email protected]]host[:P ort]/dest

If you specify only the source path and do not specify a destination path, rsync displays a list of files in the source path.
The last slash in the source path copies only the files in the directory; there is no slash, not only to copy the files in the directory, but also to copy the directory itself.
The end of the destination path has no slash and has no effect on the transmission.

! Common parameter Description
-delete Delete client files that do not exist on the server
-password-file=file Specify the location of the native rsyncd.secrets
-A,--archive archive mode, which means to transfer files recursively and keep all file attributes equal to-rlptgod
-V,--verbose verbose mode output
-Q,--quiet thin output mode
-C,--checksum turn on the check switch to force verification of file transfers
-R,--recursive subdirectories in recursive mode
-R,--relative using relative path information
-B,--backup creates a backup, that is, the old file is renamed to ~filename when the same file name exists for the purpose. You can use the--suffix option to specify a different backup file prefix.
--backup-dir store backup files (such as ~filename) in the directory.
-suffix=suffix Defining backup File prefixes
-U,--update only updates, which is to skip all the files that already exist in DST, and the file time is later than the file to be backed up. (Does not overwrite the updated file)
-L,--links reserved Soft link
-L,--copy-links to handle soft links like regular files
--copy-unsafe-links only copies links to links outside the SRC Path directory tree
--safe-links ignoring links to the SRC Path directory tree
-H,--hard-links reserved Hard link
-P,--perms maintain file permissions
-O,--owner keep file owner information
-G,--group keep file group information
-D,--devices keep device file information
-T,--times keep file time information
-S,--sparse special processing of sparse files to save DST space
-N,--dry-run reality which files will be transmitted
-W,--whole-file copy files without incremental detection
-X,--one-file-system do not cross file system boundaries
-B, the block size used by the--block-size=size test algorithm, is 700 bytes by default
-E,--rsh=command specifies data synchronization using RSH and SSH
--rsync-path=path Specify the path information for the rsync command on the remote server
-C,--cvs-exclude uses the same method as CVs to automatically ignore files to exclude files that you do not want to transfer
-F,--filter=rule load the filter rule from the specified file.
--existing only updates those files that already exist in DST without backing up those newly created files
--delete Delete those files that are not in the DST SRC
--delete-excluded also deletes those files that are excluded by the option specified by the Receive side
--delete-after transfer ends after removal
--ignore-errors Timely IO errors are also deleted
--max-delete=num deleting NUM files up to a maximum
-p,--partial retains files that are not fully transmitted for any reason, to expedite subsequent transmissions
--force forcibly delete a directory, even if it is not empty
--numeric-ids does not match the user and group ID of a number to a user name and group name
--timeout=time IP time-out, in seconds
-I,--ignore-times do not skip files that have the same time and length
--size-only when deciding whether to back up a file, just look at the file size regardless of file time
--modify-window=num determines whether the file is time-stamped with the time Stamp window, which defaults to 0
-t--temp-dir=dir creating temporary files in Dir
--compare-dest=dir also compares the files in DIR to determine if a backup is required
--progress Show Backup process
-Z,--compress compress the backed-up files during transmission
--exclude=pattern specify to exclude file modes that do not need to be transferred
--include=pattern specifies file modes that need to be transferred without exclusion
--exclude-from=file exclude files in the specified schema in file
--include-from=file does not exclude files that specify pattern matching
--version Print version Information
--address binding to a specific address
--config=file specify a different configuration file, do not use the default rsyncd.conf file
--port=port Specify a different rsync service port
--blocking-io using blocking IO for remote shells
-stats gives the transfer status of some files
--progress in the transmission of the real-time transmission process
--log-format=format specifying the log file format
--password-file=file get the password from file
--bwlimit=kbps limit I/O bandwidth, Kbytes per second
-H,--help display Help information

!rsyncd.conf configuration, press "[]" to set the module to divide the synchronization module. Each module contains a parameter definition that is formatted as name = value. Format, parameter description and common settings are as follows:
#默认存放位置/etc/rsyncd.conf
#全局参数
#指定消息文本文件, when the client connection succeeds, the contents of the file are displayed to the client
MOTD file=/usr/local/etc/rsyncdmsg

#指定pid文件
PID File=/var/run/rsyncd.pid

#指定rsync监听端口, default is 873
port=873

#指定IP
address=127.0.0.1

#模块参数, some module parameters can also be defined in the global segment for all modules.
#模块名为sync_test
[Sync_test]

#以nobody身份运行rsync Server
UID = nobody 

#指定守护程序以root方式运行时模块应当替换的文件传入和传出的组名或组ID, with the "UID" option.
GID = Nobody

#同步模块备注
Comment = Backup Demo

#需要做鏡像的目錄
Path =/opt/data

#认证用戶名, not specified to allow anonymous. Multiple user names can be separated by spaces or commas.
Auth users = Tester

#密码文件存放路径
Secrets file =/usr/local/etc/rsyncd.secrets 

#同步是否为只读, the default is Yes
Read Only = yes

#同步是否为只写, the default is no
Write Only=no

#当客户请求可以使用的模块列表时, whether the module should be listed. If you set this option to False, you can create a hidden module. The default is Yes
List=yes

#不对指定类型文件压缩
Dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz *.jpg 

#如果设为true, the file is chroot to the directory specified by path when it is transferred. You need rsync to start with root, and you cannot back up the directory files that point to external symbolic connections. The default value is true.
Use Chroot=true

#指定该模块的最大并发连接数量, the default value is 0.
Max Connections=0

#指定支持max the lock file for the connections parameter.
Lock File=/var/run/rsyncd.lock

#指定日志记录消息级别, the default is daemon.
#常见的消息级别是: Uth, Authpriv, cron, Daemon, FTP, Kern, LPR, mail, news, security, Sys-log, user, UUCP, local0, Local1, Local2, Local3,local4, LOCAL5, Local6 and LOCAL7.
Syslog Facility=daemon

#指定同步日志文件位置, do not specify to deposit logs into syslog
Log File=/var/log/rsyncd.log

#如果为true, the password file can only be accessed by a user running the rsync server, and no other user can access the file. The default value is true.
Strict modes=true

# Only allow clients that specify IP to connect to the module
# multiple IP or network segments are separated by spaces, "*" means all, and the default is to allow all hosts to connect.
# Example of network segment setup: 192.168.0.0/24, can also be 192.168.0.0/255.255.255.0
Hosts allow=*

# clients that specify IP are not allowed to connect to the module, which is not specified by default
Hosts Deny

# whether to ignore IO errors on the server, in general, Rsync will skip the--delete operation in the event of an IO error to prevent serious problems due to temporary resource shortages or other IO errors.
Ignore Errors=yes

#忽略没有访问权限的文件.
Ignore Nonreadable=yes

#用ftp格式的文件来记录下载和上载操作在单独的日志中.
Transfer logging

# custom log file fields. The format is a string that contains the format definition character
# Main Definitions and meanings:
#%h Remote Host name
#%a Remote IP address
#%l file length of characters
#%p The process ID of the rsync session
#%o operation type: "Send" or "recv"
#%f file name
#%p Module Path
#%m Module Name
#%t Current Time
#%u authenticated user name (null when anonymous)
# Number of bytes actually transferred by%b
#%c When a file is sent, the field records the checksum of the file
#默认log格式为: "%o%h [%a]%m (%u)%f%l", Generally, "%t [%p]" is added to the head of each line.
Log Format=%o%h [%a]%m (%u)%f%l

# Set the Sync timeout time. The unit is seconds, and 0 means there is no timeout definition, which is also the default value.
timeout=100

# list prohibits the list of command parameters used by the client. The full name of the command must be used.
Refuse options

# used to specify multiple files or directories separated by spaces (relative paths) and add them to the Ignore list. Only one exclude option can be specified for a module.
# but one thing to keep in mind is that this option has some security issues, and customers are likely to bypass the exclude list, which is best combined with the UID/GID option if you want to ensure that a particular file cannot be accessed.
Exclude

# Specifies a file name that contains the definition of the exclude pattern, and the server reads the Exclude list definition from the file.
Exclude from

# used to specify that files or directories that meet the requirements are not excluded.
Include

# Specifies a file name that contains the definition of the include pattern from which the server reads the Include list definition.
Include from


!rsyncd.secrets configuration, format user name: password, each line specifies a user.
Example: tester:123456
Rsyncd.secrets file to modify the permissions to 600, or the synchronization will be error.

Filter rules for!rsync
Rsync establishes a sequential list in the order of the filter rules in the command line. The syntax for the filter rule is as follows:
Rule [Pattern_or_filename]
Rule,modfiers [Pattern_or_filename]
You can use the full rule name, or you can use the shorthand name. If you use shorthand, the comma between rule and modefiers in the preceding syntax is optional, followed by the pattern or filename (if present) must have a space or underscore.
If the rule is read from a file, the blank line in the file is ignored and lines beginning with # are treated as comments.

The following rule is available:
Exclude,-: Exclude mode
Include, +: include mode
Merge,. : Specifies a merge-file for multiple rules to read
Dir-merge,:: Specify a per-directory merge-file
Hide, H: Specifies a pattern in which files that conform to the pattern will be hidden to prevent transmission
Show, S: Do not hide files that conform to this pattern
Protect, P: Specify a pattern to prevent files from being deleted
Risk, R: Files that conform to this pattern are not protected
Clear,! : Clears the current list of include/exclude modes (this option has no parameters)

Exclude and include two rule support modfiers, available modfiers as follows:
/, specify the Include/exclude rule to match the absolute path of the current project. For example,-//etc/passwd to exclude a password file whenever a file is transferred from the/etc directory.
!, tell rsync that the include/exclude rule will take effect when the pattern match fails. Like,--! */, it will match all non-catalog files.
C, the modifier indicates that all global cvs-exclude rules will be inserted into the place of-C. There are no arguments behind the modifier.
S, which indicates that the rule acts on the sending side. When a rule is in effect on the sender, it prevents the file from being transmitted. This modifier is typically used for rules that are in effect at both ends, unless--delete-excluded is specified, which will make the rule only effective at the sending side by default. Another way to specify the send-side includes/excludes is to use Hide (H) and Show (S).
R, usually used to indicate that the rule applies to the receiving end. When a rule is in effect on the receiver, it prevents the file from being deleted. Another way to specify the receive-side includes/excludes is to use Protect (P) and Risk (R) rules.

Each--filter 、--include 、--exclude option accepts only one rule/pattern, and if you want to add multiple Rule/pattern, you can repeat these options on the command line, or use the Merge-file syntax in the--filter option, or use the--include-from/--exclude-from option.
--include and--exclude are simplified versions of the--filter option.
Daemon filter chain consists of "filter", "Include from", "include", "Exclude from", "exclude" parameters, the first matching mode will take effect.

The matching principle of!rsnyc
1 if "/" appears at the beginning of the pattern, then it marks a special position in the hierarchy, otherwise it is just the end of the matching path.
Therefore, "/foo" will match Foo (for global rules) in "Root of the transfer", or Foo (for per-directory rules) in the Merge-file directory.
The unqualified foo will match foo at any location in the file system, because the algorithm takes effect recursively from top to bottom, as if each part of the path turns into the end of a file or directory.
For example FOO/A/B/C, the algorithm will interpret the path is FOO/A,FOO/A/B,FOO/A/B/C, the algorithm in turn A, B, c as a file or directory end. In fact, a non-anchored "Sub/foo" will match Foo at any location that contains sub-directories in the hierarchy.

2 If "/" appears at the end of the pattern, it matches only the directory, not the regular file, link, or device.

3 Rsync Checks if the pattern contains the following wildcard characters to determine whether to make a simple character match or a wildcard match:
*: Matches any part of the path and encounters a slash termination
* *: matches anything, including slashes
? : matches any single character except the slash ("/")
[: Matches a character set, such as [A-z], or [[: Alpha:]]

4 in wildcard mode, the backslash ("\") escapes the wildcard character, and if the wildcard does not exist, it is interpreted as a normal

5 If the pattern contains "/" (trailing "/" is not counted) or "* *", it will match the full path, including the leading directory (i.e., foo/a, which matches both a and the leading Foo), and if the pattern does not contain "/" or "* *", it only matches the last part of the path. Note: The algorithm is applied recursively, so the actual "full path" may be from the starting directory down to any part of the path.
6 The pattern ending in dir_name/*** matches both the directory (as specified by dir_name/) and the matching of all files in the directory (as if dir_name/** were specified).
Note: If the-r option is used (the-a option implies this option), then, from top to bottom, each part of the path is accessed, so the Include/exclude mode recursively takes effect for each component of the path (for example, to include/foo/bar/baz, you cannot exclude/ Foo and/foo/bar).
When rsync looks for the file to be sent, the exclude mode is actually a short-circuit of rsync in the calendar directory. If a pattern excludes a particular parent directory, it can make a deeper include mode unavailable, because rsync cannot go down (matching files) through the Exclusion section in the hierarchy. That is, if the pattern excludes a specified parent directory, it cannot continue to match subdirectories or files under that parent directory.


The Rsync filtering and matching unit focuses on Rsync III: Filtering rules , we strongly recommend that you study carefully.

(lykyl original, reproduced please specify the source)

A Concise handbook of rsync