Linux system backup and Restore tool 4 (rsync/Data Sync Tool)

Rsync is a tool that can back up the system as well as data synchronization.

On Jenkins, you can use rsync to combine SSH with a password-free login for data synchronization and distribution. This allows the deployment to be fully imperative, without the need to rely on any plug-in to implement.

Command reference: Http://man.linuxde.net/rsync

Description

The rsync command is a remote data synchronization tool that enables fast synchronization of files between multiple hosts via Lan/wan. Rsync uses the so-called "rsync algorithm" to synchronize files between local and remote two hosts, which transmits only the different portions of two files, rather than the entire transfer every time, so it is quite fast. Rsync is a very powerful tool, and its commands have a lot of feature options, and we have an analysis of its options below.

Grammar

rsync [OPTION] ... SRC Destrsync [OPTION]... SRC [[Email protected]]host:D estrsync [OPTION]... [[email protected]] HOST: SRC  destrsync [OPTION]... [[email protected]] HOST::SRC DESTrsync [OPTION] ... SRC [[Email protected]]host::DESTrsync [OPTION] ... rsync://[[email protected]]host[:P ort ]/SRC [DEST]

There are six different modes of operation for Rsync, which corresponds to the above six command formats:

1. Copy the local file. This mode of operation is initiated when both the SRC and des path information do not contain a single colon ":" delimiter. Such as:rsync -a /data /backup
2. Use a remote shell program (such as rsh, SSH) to copy the contents of the local machine to the remote machine. This mode is started when the DST path address contains a single colon ":" delimiter. Such as:rsync -avz *.c foo:src
3. Use a remote shell program (such as rsh, SSH) to copy the contents of the remote machine to the local machine. This mode is started when the SRC address path contains a single colon ":" delimiter. Such as:rsync -avz foo:src/bar /data
4. Copy files from the remote rsync server to the local machine. This mode is started when the SRC path information contains the "::" delimiter. Such as:rsync -av [email protected]::www /databack
5. Copy files from the local machine to the remote rsync server. This mode is started when the DST path information contains the "::" delimiter. Such as:rsync -av /databack [email protected]::www
6. The list of files for the remote machine is listed. This is similar to the rsync transfer, but only if the local machine information is omitted from the command. Such as:rsync -v rsync://192.168.78.192/www

Options

-V,--verbose verbose mode output. -Q,--quiet thin output mode. -C,--checksum turn on the check switch to force a checksum on the file transfer. -A,--archive archive mode, which means that files are transferred recursively, and all file attributes are maintained, equal to-rlptgod. -R,--recursive subdirectories are processed in recursive mode. -R,--relative uses relative path information. -B, --BackupCreate the backup, that is, for the purpose already exists to have the same file name, the old file is renamed to ~filename. You can use the--suffix option to specify a different backup file prefix. --Backup-dirThe backup file (such as ~filename)stored in the directory. -suffix=SUFFIX defines the backup file prefix. -U,--update only updates, that is, skips all existing DST, and the file time is later than the file to be backed up and does not overwrite the updated file. -L,--links keep the soft link knot. -L, --Copy-links to handle soft links like regular files. --Copy-unsafe-links only copies links to the SRC Path directory tree. --safe-links ignores links to links other than the SRC path directory tree. -H,--hard-links preserves hard links. -P,--perms maintain file permissions. -O,--owner Keep the file owner information. -G,--group maintain file group information. -D,--devices Keep the device file information. -T,--times Keep file time information. -S,--sparse Special processing of sparse files to save DST space. -N,--dry-run Reality which files are to be transmitted. -W,--whole-file copy files without incremental detection. -X,--one-file-system do not cross file system boundaries. -B,--block-size=the size test algorithm uses a block size that defaults to 700 bytes. -E,--rsh=CommandSpecifies that data is synchronized using RSH and SSH. --rsync-Path=PATHSpecifies the path information for the rsync command located on the remote server. -C,--cvs-exclude uses the same method as CVs to automatically ignore files, which are used to exclude files that you do not want to transfer. --existing only updates those files that already exist in DST, without backing up those newly created files. --delete Delete those files that are not in the DST Src. --delete-excluded also deletes those files that are excluded by the option specified by the Receive side. After the--delete-after transmission is finished, delete again. --ignore-errors Timely IO errors are also deleted. --max-delete=num Deletes a maximum of num files. --partial retains files that are not fully transmitted for any reason, in order to expedite subsequent transmissions. --force forcibly deletes the directory, even if it is not empty. --numeric-ids does not match the user and group ID of the number to the user name and group name. --timeout= Timethe IP time-out, in seconds. -I.,--ignore-times does not skip files that have the same time and length. --size-only when deciding whether to back up a file, just look at the file size regardless of the file time. --modify-window=NUM Determines whether the file is time-stamped with the time Stamp window, which defaults to 0. -T--temp-dir=DIRCreate a temporary file in Dir. --compare-dest=DIRalso compare the files in dir to determine if a backup is required. -P is equivalent to--partial. --progress shows the backup process. -Z,--compress The files that are backed up are compressed at the time of transmission. --exclude=pattern Specifies to exclude file patterns that do not need to be transferred. --include=Pattern Specifies the file mode that is not excluded and that needs to be transferred. --exclude-from=file excludes files in the specified schema in file. --include-from=file does not exclude files that specify pattern matching. --version print version information. --address is bound to a specific address. --config=FILE specifies a different configuration file and does not use the default RSYNCD.conf file. --port=Port Specifies the other rsync service ports. The--blocking-io uses blocking IO on the remote shell. -stats gives the transfer status of some files. --progress the actual transmission process during transmission. --log-format=FormATSpecifies the log file format. --password-file=file gets the password from file. --bwlimit=kbps Limit i/o bandwidth, Kbytes per second. -H,--help displays help information.

Instance

SSH mode

Start the SSH service on the server first:

Start start sshd: [OK]

Synchronizing with Rsync

Next, you can use the rsync command on the client side to back up the data on the server, and SSH is backed up by the system user, as follows:

rsync-vzrtopg--progress-e ssh--delete [email protected]:/www/*/databack/experiment/  rsync[email protected]172.16.78.192files to considertest/a0 100% 0.00kb/s 527:35:41 (1, 20.0% of 5)b, 100% 65.43kb/s 0:00:00 (2, 40.0% of 5)C0 100% 0.00kb/s 527:35 : (3, 60.0% of 5)DD100663296 100% 42.22mb/s 0:00:02 (4, 80.0% of 5)Bytes received 98190 bytes 115 63.06 bytes/100663363 speedup is 1024.19

The information above describes the overall backup process and the size of the total backup data.

Background service mode

Start the Rsync service, edit the /etc/xinetd.d/rsync file, change it to disable=yes disable=no , and restart the XINETD service, as follows:

vi/etc/xinetd.d/rsync  off# description:the rsync server is a good addition to an FTP server, as it # allows CRC Checksumming etc. /c6>=== = =/usr/bin/= + = USERID} 

/etc/init.d/  xinetd Restart stop xinetd: [OK] start xinetd: [OK]

Description: Ubuntu is not necessarily in this location, the above example is based on CentOS.

Creating the configuration file, after installing the Rsync program by default, does not automatically create the Rsync master profile, it needs to be created manually, its master profile is "/etc/rsyncd.conf", the file is created, and the following is inserted:

  VI/ETC/RSYNCD.    conf  uid  =rootgid  =< Span style= "color: #000000;" >rootmax connections  =4log file  =/var/log/rsyncd. logpid file  =/var/run /rsyncd.pidlock file  =/var/run /rsyncd. locksecrets file  =/etc/rsyncd. passwdhosts deny  =172.16.78.0/22[www]comment  = backup   Web  path  =/ Noexclude  = testauth users  =work 

Create a password file, in this way can not use the system user authentication to the client, so you need to create a password file, the format is "Username:password", the user name and password can be arbitrarily defined, it is best not to be consistent with the system account, At the same time to create the password file permissions set to 600, which in the previous module parameters are described in detail.

Echo "Work:abc123" >/etc/rsyncd.  600/etc/rsyncd.passwd

Backup, complete the above work, you can now back up the data, as follows:

  rsync-avz--progress--delete [Email protected]::  www/databack/experiment/rsync   password:receiving file list  ... 6 files   to consider ./files  ... a  0 100% 0.00kb/s 528:20:41 (1, 50.0% of 6) b  100% 65.43kb/s 0:00:00 (2, 66.7% of 6) c  0 100% 0.00kb/s 528:20:41 (3, 83.3% of 6) DD  100663296 100% 37.49mb/s 0:00:02 (4, 100.0% of 6) sent  172 Bytes Received 9 8276 bytes 17899.64 bytes/sectotal size is  150995011 speedup is 1533.75 

Recovery, when the server data problems, then need to through the client's data on the service side of the recovery, but only if the service side allows the client to have write permission, or the client can not directly restore the service side, using rsync to restore the data method is as follows:

Rsync-avz--progress/databack/experiment/rsync/[email protected]::wwwfiles to consider./ab100% 0.00kb/s 0:00:00 (2, 66.7% of 6)258 bytes received 95.43 bytes bytes/150995011 speedup is 452080.87

Reference:

Http://wangchujiang.com/linux-command/c/rsync.html (the above content is transferred from this article)

Http://blog.csdn.net/zpf336/article/details/51659666 (example)

https://my.oschina.net/jccpp/blog/124941 (example)

http://davidbj.blog.51cto.com/4159484/1181388/(example)

http://linuxcentos.blog.51cto.com/7176396/1213241 (example)

https://yq.aliyun.com/ziliao/120031 (example)

http://leyewen.blog.163.com/blog/static/2506061420130249419837/(example)

Http://www.cnblogs.com/Gbeniot/p/5482366.html (example)

Linux system backup and Restore tool 4 (rsync/Data Sync Tool)