Linux uses rsync remote file synchronization and backup tools teaching and learning examples __linux

rsync Simple

The role of rsync is like the general Linux CP and SCP directive, which can be copied from the source location to the destination location, however, rsync is more efficient than CP and SCP in a copy of the file, and supports both the connection and the devices, as well as the retention of the file's owner, group Group and permissions,

In the first copy of the case, rsync will copy the full file, and then again, the Delta transfer algorithm is used to check the differences between the new files, only to transfer the part of the change, to speed up the backup speed, especially when the big file is exhausted, the effect is more obvious. In addition, Rsync also supports the automatic compression and decompression of the data when using the Internet, which can effectively reduce the consumption of network bandwidth. Install rsync

Most Linux releases have built-in rsync tools, and if your system is not installed, you can usually install it through a system suite. The Red Hat series of Linux can be Yum installed:

sudo yum install rsync

The Debian series of Linux can be apt-get:

sudo apt-get install rsync

basic usage of rsync

The basic language of rsync is structured as follows:

Rsync Parameters Source File file

Here are a few of the most common parameters: the-v:verbose model, which outputs more detailed messages. -R: Recursive is equipped with all subdirectories and files. -A: The enclosure mode, as in the case of-rlptgod, is to be transferred to all subdirectories and files, retaining the link, the owner of the file, the group, the rights, and the time stamp recording. -Z: To activate compression. -H: The numbers are output in a more easily read format.

The simplest use of rsync is to replicate the local file:

RSYNC-AVH myfile.gz/home/pi/tmp/

Sending incremental file list
myfile.gz

sent 14.34M bytes  received bytes  28.67M bytes/sec
Total The size is 14.33M  speedup is 1.00

The effect is similar to that of Cp-r, which can be myfile.gz to/home/pi/tmp/, but if you perform the second time, rsync will automatically skip the files that have not changed:

RSYNC-AVH myfile.gz/home/pi/tmp/

Sending incremental file list

sent bytes received of bytes 172.00 bytes/sec Total  size is
14.33M  Speedup is 166,658.15

This usage is appropriate for both the file and the catalogue:

rsync-avh/path/to/myfolder/home/pi/tmp/

rsync Remote Backup

Rsync can also be used for remote backup between different machines, which is similar to the SCP instruction, but rsync is more efficient:

Rsync-avzh/mypath/myfile.gz pi@192.168.1.12:/mybackup/

This will bring the local myfile.gz back to Pi@192.168.1.12 's/mybackup/, and when it comes to this remote backup, rsync will log in to the remote machine in SSH, so after the run of the backup instructions, it's OK. To enter the pi@192.168.1.12 password, you will then start to prepare the data, and the output will look like this:

pi@192.168.1.12 ' s password: 
sending incremental file list
myfile.gz

sent 13.62M bytes received  34 Bytes  48.56K bytes/sec Total
size are 14.33M  speedup is 1.05

And here we add a-Z-parameters, so that rsync can automatically decompress the data and then send it back, and then, when it receives data from the far side, it automatically shrinks and reduces the amount of network traffic.

Rsync can also be able to back up the remote file to the local side, and the language is similar to the SCP:

Rsync-avzh pi@192.168.1.12:/mypath/myfile.gz/mybackup/

pi@192.168.1.12 ' s password: 
receiving incremental file list
myfile.gz sent-bytes received  23.74M Bytes  571.98K bytes/sec Total
size are 24.14M  speedup is 1.02

In this case of rsync, since we added the-a parameters, so it can be used in a file or a whole catalogue, as in the case of Scp-r, and since rsync only has a variable part of the movement, it's usually used in this way when it comes to data. Restrict Network bandwidth

If you don't want rsync to use too much network bandwidth to affect normal service through network backup, you can add--bwlimit parameters to specify the speed limit of the data transfer:

Rsync-avzh--bwlimit=100k pi@192.168.1.12:/mypath/myfile.gz/mybackup/

pi@192.168.1.12 ' s password: 
receiving incremental file list
myfile.gz sent-bytes received  14.34M Bytes  99.22K bytes/sec Total
size are 14.33M  speedup is 1.00

Customize SSH Connection Port

The connection port (port) Number of the normal SSH remote login service is 22, but some people are protecting the server from too much cyber attacks, and will change the SSH connection to another number, for example, 12345, but in this way, when using rsync for remote backup data, You will also need to follow the designation of the connection port number.

Suppose 192.168.1.12 This server has an SSH service connection of 12345, below is an example of the data backup data from Rsync:

Rsync-avzh-e ' ssh-p 12345 '/mypath/myfile.gz pi@192.168.1.12:/mybackup/

Here we add a-e reference, which is used to specify the instructions to use for the remote login. The preset directive is SSH, and here we change the instruction to Ssh-p 12345, which means using the 12345 port to login ssh (please refer to the SSH Directive's-p reference usage). showing the degree of transfer

If you want rsync to be able to show progress at the time of the transfer, you can add--progress parameters:

Rsync-avzh--progress pi@192.168.1.12:/mypath/myfile.gz/mybackup/

This shows the progress of the transfer, the speed of the transfer, and the rest of the time in the process of preparing each file:

pi@192.168.1.12 ' s password: 
receiving incremental file list
myfile.gz
         24.14M 100%    623.52kb/s 0:00:37 (xfr#1, to-chk=0/1)

sent bytes received  23.74M bytes  558.52K bytes/sec Total
size is 24.14m< C9/>speedup is 1.02

Sync Delete File

The Rsync preset will only synchronize the source-side files to the destination (sync all new or modified files), but if a file is removed from the source, rsync will not automatically delete the destination file, so that when the data is not removed, the backup file will not be erased as well.

If you want to allow rsync to sync, you can add--delete parameters to the file that does not exist at the source, and if there is no source file that is new and not less, it is the same as the general copy:

RSYNC-AVH--delete myfolder/backup/

Sending incremental file list
./
data1.txt
data2.txt
data3.txt data4.txt sent

432 Bytes  received bytes  1.05K bytes/sec Total
size are 116  speedup is 0.22

This time, if we're going to delete the Data1.txt and data2.txt from the source file, and add the data5.txt, we'll do an rsync:

RSYNC-AVH--delete myfolder/backup/

Sending incremental file list
deleting data2.txt
deleting Data1.txt
./
data5.txt

sent 190  received bytes  508.00 bytes/sec Total
size is speedup is  0.34

At this point, rsync synchronizes the Data1.txt and data2.txt at the end of the backup, and adds Data5.txt at the same time.

If we don't add--delete parameters here, Rsync will only add data5.txt and not delete any files. Backup-specific files

Let's assume that our files and our catalogue are as follows:

Tree MyFolder

MyFolder
├──chinese.py
├──data1.txt
├──data2.txt
├──find_edimax.c
    └──src PACK.C
    ├──test1.txt
    └──test2.txt

To allow rsync to exclude all *.txt files in the backup file, you can use the--exclude parameters:

RSYNC-AVH--exclude ' *.txt ' myfolder/backup/

Sending incremental file list
./
chinese.py
find_edimax.c
src/
src/pack.c

sent 3.91K bytes  received bytes  7.99K bytes/sec Total
size was 3.59K  speedup is 0.90

We can use multiple--exclude to exclude multiple files, such as:

RSYNC-AVH--exclude ' *.txt '--exclude ' *.py ' myfolder/backup/

Sending incremental file list
./
find_edimax.c
src/
src/pack.c

sent 3.74K bytes  Received bytes  7.61K bytes/sec Total
size are 3.50K  speedup is 0.92

If you only want to have some specific files, you can use--exclude with--include, for example, just a copy of all the *.C's C-language codes:

RSYNC-AVH--include ' *.c '--include '--exclude ' * ' myfolder/backup/

Sending incremental file list
./
find_edimax.c
src/
src/pack.c

sent 3.74K bytes  Received bytes  7.62K bytes/sec Total
size was 3.50K  speedup is 0.92

Here we add two--include to specify the files to be prepared, *.C is the original file that contains all C languages, and the other one means that all the records are included, and if there is no reference to the record, all the records will be excluded from the--exclude. The resulting *.c in all subdirectories are also excluded. Finally, add a--exclude to exclude all remaining files, please note that--exclude should be placed after--include, the order can not be adjusted.

limit the size of the backup file

Rsync can also select a backup file according to the size of the file, assuming that the following files are available in the MyFolder catalogue:

Ls-l myfolder/

Total 15632
-rw-r--r--1 pi pi  1658348 Feb  5 09:09 bluez-5.43.tar.xz
-rw-r--r--1 pi pi       Feb  5 0 7:57 chinese.py
-rw-r--r--1 pi pi     2736 Feb  5 07:57 find_edimax.c
-rw-r--r--1 pi pi 14332601 Feb  5 09:09 myfile.gz
-rw-r--r--1 pi pi      763 Feb  5 08:02 pack.c

--min-size can specify the minimum size of a backup file, such as a file with a backup of more than 1MB:

RSYNC-AVH--min-size=1m myfolder/backup/

Sending incremental file list
./
bluez-5.43.tar.xz
myfile.gz

sent 16.00M bytes  received-bytes  31.99M bytes/sec Total
size are 15.99M  speedup is 1.00

The--max-size can specify the maximum size of the backup file, such as a file with only 4KB below:

RSYNC-AVH--max-size=4k myfolder/backup/

Sending incremental file list
./
chinese.py
find_edimax.c
pack.c

sent 3.91K bytes  Received bytes  7.97K bytes/sec Total
size are 15.99M  speedup is 4,012.68

--min-size and--max-size can also be used at the same time, such as a file with 1KB to 2MB only:

RSYNC-AVH--min-size=1k--max-size=2m myfolder/backup/

Sending incremental file list
./
bluez-5.43.tar.xz
find_edimax.c

sent 1.66M bytes  received Bytes  3.32M bytes/sec Total
size are 15.99M  speedup is 9.62

Auto Erase source file

If you want to allow rsync to delete the source files (that is, the effect of MV) after backup, you can add--remove-source-files parameters:

RSYNC-AVH--remove-source-files myfolder/backup/

Sending incremental file list
./
bluez-5.43.tar.xz
chinese.py
find_edimax.c
myfile.gz
Pack.c

sent 16.00M bytes  received 154 bytes  10.67M bytes/sec Total
size are 15.99M  speedup is 1.00

After doing so, the myfolder/will be emptied and all the information would be moved to the backup/, so be careful and don't delete important files. Test rsync Parameters

If a beginner does not determine whether their rsync parameters are correct, you can add--dry-run before you actually perform a test, plus this one will still produce a normal message, but it won't move to any file in any of these cases:

RSYNC-AVH--dry-run--remove-source-files myfolder/backup/

Sending incremental file list
bluez-5.43.tar.xz
chinese.py
find_edimax.c
myfile.gz
pack.c

sent 194 bytes  received bytes  450.00 bytes/sec Total
size are 15.99M speedup is  71,086.85 (DRY RUN)

This makes it easy for users to check whether their parameters are properly configured. crontab Regular Backup

Typically, if you want to make a backup on the local side, you can periodically perform this instruction in crontab and periodically back up important information to the specified catalogue:

# m H  Dom Mon Dow   command
0 5 * * 1 rsync-a/path/to/folder/path/to/backup/

This system will perform the rsync backup file at 5 points per week of the morning. Update only existing files

If, in the case of a file, you only want to update the past and have a copy of the file, remove the new files, you can use the--existing parameters.

Let's assume that we've already myfolder/the file to backup/:

RSYNC-AVH myfolder/backup/

And this time, a new file is added:

Touch Myfolder/new.file

If we just update the existing files in the backup/and eliminate the new additions, we can use the--existing parameters:

RSYNC-AVH--existing myfolder/backup/

Sending incremental file list
./

sent 201 bytes Received-bytes 440.00 bytes/sec Total  size is
15.9 9M  speedup is 72,702.46

showing file change information

When you perform rsync, add-I parameters can be used to change information about a file:

Rsync-avhi myfolder/backup/

Sending incremental file list
. D. T....... /
... f...p ... find_edimax.c
>f ... T ... myfile.gz
>f+++++++++ new.file
>f.st ... pack.c sent

14.34M bytes received  79. Bytes  3.19M bytes/sec Total
size are 15.99M  speedup is 1.12

After  -i , each file will have an extra label before each item, and the string has 11 bars, which are divided into  yxcstpoguax, meaning the following: y:<  on behalf of the file sent to the remote,>   on behalf of the file to the local side,c  on behalf of the local change (set up a record, etc.),h  on behalf of hard links (hard link),.  representative does not change,*  represents the rest of the fields contain information (for example   deleting). X: File type,f  as a general file,d ,l  for the connection,d  as a backup file (device),s  as a special file (such as Sockets or FIFO). C: There is a change in the file's contents. S: There is a change in the size of the file. T: On behalf of the file, the timestamp changes. P: There is a change in the rights of the file. O: There is a change in the person who owns the file. G: On behalf of the file group there are changes. U: Keep the position. A: There is a change in the file ACL information. X: On behalf of file expansion (extended attribute) there is a change.