Linux rsync Sync command (worth collecting) _linux shell

If you are a maintenance engineer, you are likely to face dozens of, hundreds of or even thousands of servers, in addition to bulk operations, environmental synchronization, data synchronization is also essential skills.

When it comes to "sync," the weapon that has to be mentioned is rsync, and today is the art of sync I see in this tool.

[With no options]

We often use rsync in this way:

Copy Code code as follows:

$ rsync main.c Machineb:/home/userb

1 data synchronization is triggered as long as the destination file content is different from the source, and rsync ensures that the file content is the same on both sides.

2 but rsync will not sync the file "Modify time", whenever there is data synchronization files, the destination of the file "modify" will always be modified to the latest moment.

3 rsync will not focus too much on the rwx permissions of the destination file, and if the destination does not have the file, the permissions will remain consistent with the source, and if the destination has this file, the permissions will not change with the source side.

4 as long as rsync has read access to the source file and has write access to the target path, rsync can ensure that the destination files are synchronized to the source side.

5 rsync can only create files with the login destination account, and it does not have the ability to keep the destination file in the same group and source. (unless you are using root authority, you are eligible to require consistency of the subject, consistent group)

[-t option]

We often use the-t option in this way:

Copy Code code as follows:

$ rsync-t main.c Machineb:/home/userb

1 with the-t option, rsync will always think of one thing, which is to sync the source file "Modify time" to the target machine.

2 with the-t option, rsync will become smarter, comparing the timestamps and file sizes of both sides of the file before synchronizing, and if so, the file will no longer be updated if it is consistent.

3 because of the cleverness of rsync, it will be mistaken for cleverness. If the destination file has a consistent timestamp, size, and source, but the content happens to be inconsistent, rsync is not found. This is the legendary "pit"!

4 for Rsync Smart, the solution is to use the-I option.

[-I option]

We often use the-I option in this way:

Copy Code code as follows:

$ rsync-i main.c Machineb:/home/userb

The 1-i option will make rsync very well-behaved and honest, and it will start data synchronization by file.

The 2-I option ensures that the data is consistent at a cost that slows down as we discard the quick check policy. (Quick check policy, that is, first look at the file timestamp and file size, in order to exclude a group of files that are the same)

3 Regardless of the situation, the modify time of the destination file is always updated to the current moment.

"-V Option"

This option, easy to understand, is to let rsync output more information, we can give an example:

$ rsync-vi main.c machineb:/home/userb                          
main.c
sent ba bytes received bytes 246.00 bytes/sec total
size is One speedup is 0.09

The more you add V, the more log information you can get.

$ rsync -vvvvt abc.c machineB:/home/userB 
cmd= machine=machineB user= path=/home/userB
cmd[0]=ssh cmd[1]=machineB cmd[2]=rsync cmd[3]=--server cmd[4]=-vvvvte. cmd[5]=. cmd[6]=/home/userB 
opening connection using: ssh machineB rsync --server -vvvvte. . /home/userB 
note: iconv_open("ANSI_X3.4-1968", "ANSI_X3.4-1968") succeeded.
(Client) Protocol versions: remote=28, negotiated=28
(Server) Protocol versions: remote=30, negotiated=28
[sender] make_file(abc.c,*,2)
[sender] flist start=0, used=1, low=0, high=0
[sender] i=0 abc.c mode=0100664 len=11 flags=0
send_file_list done
file list sent
send_files starting
server_recv(2) starting pid=31885
recv_file_name(abc.c)
received 1 names
[receiver] i=0  abc.c mode=0100664 len=11
recv_file_list done
get_local_name count=1 /home/userB
recv_files(1) starting
generator starting pid=31885 count=1
delta transmission enabled
recv_generator(abc.c,0)
abc.c is uptodate
generate_files phase=1
send_files phase=1
recv_files phase=1
generate_files phase=2
send files finished
total: matches=0 hash_hits=0 false_alarms=0 data=0
generate_files finished
recv_files finished
client_run waiting on 14318
sent 36 bytes received 16 bytes 104.00 bytes/sec
total size is 11 speedup is 0.21
_exit_cleanup(code=0, file=main.c, line=1031): entered
_exit_cleanup(code=0, file=main.c, line=1031): about to call exit(0)

[-Z option]

This is a compression option, and as long as this option is used, rsync compresses and transmits the outgoing data first. Recommended for scenarios with poor network environments.

In general, the compression algorithm for-Z will be the same as gzip.

[-R Option]

The first time we use rsync, we often encounter such a dilemma:

$ rsync Superman machineb:/home/userb
skipping directory Superman

If you don't tell rsync that you need it to help you sync folders, it's not going to take the initiative, which is where rsync is lazy.

So, if you really want to sync a folder, then add the-r option, which is recursive (recursive, loop), like this:

$ rsync-r Superman Machineb:/home/userb

As we said in the above tutorial, if the timestamp and file size are identical, only the file content is different and you do not use the-i option, then rsync will not synchronize data.

So, ask a question: "Because in the Linux world, folders are files, and if such files (folders) are only different, and the timestamp and file size are the same, will rsync find out?" ”

Experiment you can do it yourself, the conclusion is here to tell you:

For folders, rsync will be very discerning, as long as you add the-r option, it will be dedicated to the folder to check, and not only the folder itself do "Quick check".

[-l option]

If we were going to sync a soft link file, what would you guess rsync would suggest?

$ ll Total
128
-rw-rw-r--1 UserA usera Dec 07:00 abc.c lrwxrwxrwx
1 UserA usera 5 Dec 11:35 Softlink -> abc.c
$ rsync softlink machineb:/home/userb
skipping non-regular file "Softlink"

Well, you guessed right, and Rsync rejected US mercilessly. Once it finds out that a file is a soft link, it ignores it unless we add the-l option.

Copy Code code as follows:

$ rsync-l Softlink Machineb:/home/userb

With the-l option, rsync completely maintains the soft link file type, copying the soft link file to the destination, without "follow link" to the entity file.

If I just wanted rsync to take follow link, I would use the-l option. You can try the effect yourself.

[-P option]

The full name of this option is "Perserve permissions", as the name implies, is to maintain permissions.

If you don't use this option, rsync does this to deal with permissions issues:

1 If the destination does not have this file, the destination file's permissions are kept consistent with the source at the end of synchronization;

2 If the destination already exists, the file contents are synchronized and the permissions remain unchanged.

If you use the-P option, Rsync keeps the destination consistent with the source-side permissions anyway.

[-G option and-o option]

The two options are a pair that keeps the file's group and owner (owner) in a clear and unambiguous role. However, it is important to note that the change of the owner and group is often only possible with administrator privileges.

[-D option]

The-D option, which is interpreted as "preserve devices (root only)", literally means keeping the original information of the device file. Since the blogger has not actually experienced its benefits, there is not much to say.

[-A option]

The 1-A option is one of the more overbearing options in rsync because you use the-a option, which is equivalent to using the-rlptgod option. With an enemy seven, the only-a option is also. (after reading the previous article, you should be able to easily understand the role of these seven options)

The scientific name for the 2-A option should be called archive option, which is called the archive options. Using the-a option indicates that you want to take a recursive approach to synchronization and to keep all aspects consistent as much as possible.

3 But the-a option also has the Achilles heel, which is--------------A cannot sync the hard link situation. If you have this requirement, add the-H option.

[--delete option 、--delete-excluded options and--delete-after options]

All three options are related to "delete":

1–delete: If the source side does not have this file, then the destination also do not want to have, delete it. (If you use this option, you must go with the-r option)

2–delete-excluded: Specifically specify some files to be deleted at the destination.

3–delete-after: By default, rsync cleans the destination files before starting data synchronization, and if you use this option, rsync synchronizes the data before deleting the files that need to be cleaned up.

See so many delete, do you have a little liver fibrillation? Indeed, there is one such remark in the official statement of Rsync:

This option can be dangerous if used incorrectly!
It is a very good idea to run the dry run option
(-N) to what files would is deleted to make sure
Important files aren ' t listed.

In this sentence, we have learned a little trick, that is the-n option, it is a scare option, it will be affected by the list of files to warn you, but not really to delete, which gives us a confirmation of the opportunity and manoeuvre. Let's take a look at the actual usage:

$ rsync-n--delete-r. machineb:/home/userb/
deleting superman/xxx
deleting main.c
deleting

[--exclude options and--exclude-from options]

If you do not want to sync something to the destination, you can use the –exclude option to hide, rsync still attaches great importance to everyone's privacy, you can use the –exclude option many times to set a lot of "privacy."

If you want to hide too much privacy, set in the command-line option is more cumbersome, rsync is still very considerate, it provides the –exclude-from option, so you can put the privacy one by one in a file, and then let rsync directly read the file is good.

[--partial Options]

This is the legendary extension of the breakpoint function. By default, rsync deletes files that have been disconnected from the transport and then transmits them again. But in some special cases, we do not wish to retransmit, but to continue to preach.

When we are in use, we often see someone using the-p option, which is actually designed to be lazy. In the past, people always have to manually write –partial–progress, feel too laborious, rather than with a new option to replace, so-P came into being. Some readers will ask –partial I know the effect, but –progress is what to use it? Why is it that so many people want to use it, and it has so much appeal? (Really ... ）

[--progress Options]

With this option, rsync shows the progress information, what's the use, and rsync gives an interesting explanation:

This is gives a bored user something to watch.

Well, wrote so much, we see is very boring, to the actual use of –progress to solve boredom, is a good choice ^_^

Here is a detailed explanation of the rsync parameters:

-V,--verbose verbose mode output
-Q,--quiet thin output mode
-C,--checksum Open the check switch to force the file transfer to verify
-A,--archive archive mode, which represents the transfer of files recursively and maintains all file attributes equal to-rlptgod
-R,--recursive the subdirectory in recursive mode
-R,--relative using relative path information
-B,--backup creates a backup, which means that the old file is renamed to ~filename for the same file name already exists for the purpose. You can use the--suffix option to specify different backup file prefixes.
--backup-dir store backup files (such as ~filename) in the directory.
-suffix=suffix defines the backup file prefix
-U,--update only updates, skipping all files that already exist in DST, and the file time is later than the file you want to back up. (Do not overwrite the updated file)
-L,--links retention soft link knot
-L,--copy-links to treat soft links like regular files
--copy-unsafe-links only copies links that point beyond the src Path directory tree
--safe-links ignores links to the SRC path tree
-H,--hard-links keep hard links
-P,--perms keep file permissions
-O,--owner keep file owner information
-G,--group keep file group information
-D,--devices maintain device file information
-T,--times keep file time information
-S,--sparse special handling of sparse files to save DST space
-N,--dry-run which files will be transmitted
-W,--whole-file copy files without incremental detection
-X,--one-file-system do not cross file system boundaries
-B, the block size used by the--block-size=size test algorithm is 700 bytes by default
-E,--rsh=command specifies the use of rsh, SSH mode for data synchronization
--RSYNC-PATH=PATH Specifies the path information for the rsync command on the remote server
-C,--cvs-exclude use the same method as CVs to automatically ignore files to exclude files that you do not want to transfer
--existing updates only those files that already exist in DST, not the newly created files
--delete Delete files that are not in the DST SRC
--delete-excluded also deletes the receiving end of files that are excluded by the option specified
--delete-after after transmission is over
--ignore-errors in time IO errors are also deleted
--max-delete=num Delete NUM files up to
--partial retains files that are not fully transmitted for some reason, to expedite subsequent transmissions
--force forcibly deletes a directory, even if it is not empty
--numeric-ids does not match the user and group IDs of numbers to user and group names
--timeout=time IP timeout time in seconds
-I,--ignore-times not skip those files that have the same time and length
--size-only when deciding whether to back up a file, just look at the file size without considering the file time
--modify-window=num the time stamp window that determines whether a file is in the same time, default is 0
-t--temp-dir=dir create temporary files in dir
--compare-dest=dir also compares files in DIR to determine whether a backup is required
-P equals to--partial
--progress Display backup process
-Z,--compress compression of backed-up files on transfer
--EXCLUDE=PATTERN Specifies to exclude file modes that do not require transfer
--INCLUDE=PATTERN Specifies the file mode that is not excluded and needs to be transferred
--exclude-from=file exclude files in the specified mode in file
--include-from=file does not exclude files that match the file-specified pattern
--version Print version Information
--address bound to a specific address
--CONFIG=FILE specifies a different profile and does not use the default rsyncd.conf file
--PORT=PORT Specifies the other rsync service ports
--blocking-io blocking IO for a remote shell
-stats gives the transfer status of some files
--progress transmission process in real time
--log-format=format Specify log file format
--password-file=file gets the password from file
--bwlimit=kbps limit I/O bandwidth, Kbytes per second
-H,--help display Help information
The AZV option is used for general synchronous transport catalogs.