Rsync remote Synchronization

Rsync remote Synchronization

Rsync introduction:

Rsync is an excellent, fast, and multi-platform local or remote data image synchronization backup tool. Applicable to Unix, Linux, Windows, and other platforms.

During backup synchronization, by default, rsync uses its unique quick check algorithm to synchronize only the files or directories whose size or last modification time has changed (you can also follow the permission, master and other changes, you need to specify a parameter) or even only synchronize the changed content of a file, so you can quickly synchronize data.

Differences between rsync, cp, and scp:

Cp and scp tool copy are complete copies each time. rsync not only supports full backup, but also supports incremental copy. Therefore, rsync outperforms performance and efficiency.

Rsync method:
Rsync uses three main methods for data transmission:
1. Local Data Transmission
2. Transmission Through rcp, ssh, and other channels
3. Transmit data as a daemon

Local Data Transmission
Syntax:
Rsync [OPTION] SRC... [DEST]
SRC: source file
DEST: target file
Option parameter description: # generally, you can use-avz.
-A: Archive mode, recursively returning and retaining object attributes, equivalent to-rlptgoD
-R: recursive mode, including all files in directories and subdirectories
-L: The Symbolic Link file is still copied as a symbolic link file.
-P: Permission marker for retaining files
-T: indicates the time mark for retaining files.
-G: indicates the group tag of the object to be retained (used only by the superuser)
-O: indicates the owner mark of the file to be retained (used only by the superuser)
-D: Keep device files and other special files
-V: displays detailed information about the synchronization process (verbose ).
-Z: compress)
-H: Keep the hard connection file
-A: retain the ACL attribute information.
-- Delete: delete a file with a target location but not the original location.
-- Checksum: determines whether to skip a file based on the object's checksum.

Demo: Copy files in the/etc directory to the/tmp directory
[Root @ localhost ~] # Rsync-avz-P/etc/tmp/# The first copy takes a long time, and the output information is also large.
The second copy is very fast because there is no data update in the/etc directory.
[Root @ localhost ~] # Rsync-avz-P/etc/tmp/
Building file list...
2141 files to consider

Sent 61983 bytes encoded ed 20 bytes 124006.00 bytes/sec
Total size is 88435897 speedup is 1426.32
 
Note: rsync-avz-P/etc/tmp/Only synchronizes the content in the etc directory. The etc itself is not synchronized.
Rsync-avz-P/etc/tmp/test all the content in etc and etc to the tmp directory.
 
Data transmission through remote shell:
Syntax:
Pull: rsync [option] [USER @] HOST: SRC... [DEST]
Push: rsync [option] SRC... [USER @] HOST: DEST
[USER @] HOST: specifies the remote connection USER and HOST address for Rsync synchronization.
SRC: indicates the source, that is, the copied partition, file, or directory. A colon is used to connect to the HOST;
[DEST] for the destination partition, file, or directory

Demonstration: Make sure that the ssh channel is ready in advance
Push the local etc directory to the tmp directory. Because there is an ssh channel, no password is required.
[Root @ localhost ~] # Rsync-avz-P/etc-e 'ssh-p 22 'root@192.168.0.108:/tmp/

Back up the remote etc directory to the local tmp directory
[Root @ localhost ~] # Rsync-avz-P-e 'ssh-p 22 'root@192.168.0.108:/etc/tmp/

Daemon method:
[Root @ localhost ~] # Vim/etc/rsyncd. conf
Uid = root
Gid = root
Use chroot = no # Lock the Home Directory
Max connections = 200 # maximum number of connections
Timeout = 300 # timeout
Pid file =/var/run/rsyncd. pid
Lock file =/var/run/rsync. lock
Log file =/var/log/rsyncd. log
[Www] # module, which is used to read data
Path =/var/www/html/# remote backup path. Capture is also captured here. If you upload uid and gid, you need to use the upload permission. If you want to upload uid and gid, you must have the read permission.
Ignore errors # ignore errors
Read only = false # If read-only is false, data can be uploaded. If it is true, data cannot be uploaded.
List = false
Hosts allow = 192.168.0.0/24
Hosts deny = 0.0.0.0/32
Auth users = rbackup # authenticated user
Secrets file =/etc/rsync. password # store user and password files

Create a file for storing Authenticated Users
[Root @ localhost ~] # Vim/etc/rsync. password
Rbackup: RedHat
[Root @ localhost ~] # Chmod 600/etc/rsync. password # The permission must be 600
[Root @ localhost ~] # Ls/etc/rsync. password-l
-Rw ------- 1 root 15 12-10 10:18/etc/rsync. password

Start the service
[Root @ localhost ~] # Rsync -- daemon # -- daemon indicates that daemon is started as a daemon.
[Root @ localhost ~] # Ps-ef | grep rsync | grep-v grep
Root 7915 1 0 :20? 00:00:00 rsync -- daemon
[Root @ localhost ~] # Netstat-lnt | grep 873
Tcp 0 0 0.0.0.0: 873 0.0.0.0: * LISTEN
Tcp 0 0: 873: * LISTEN

Set automatic start upon startup
[Root @ localhost ~] # Echo "/usr/bin/rsync -- daemon">/etc/rc. d/rc. local
[Root @ localhost ~] # Tail-1/etc/rc. d/rc. local
/Usr/bin/rsync -- daemon

Client deployment
[Root @ localhost ~] # Echo "redhat">/etc/rsync. password # account not required for password only
[Root @ localhost ~] # Chmod 600/etc/rsync. password # File Permission 600
[Root @ localhost ~] # Cat/etc/rsync. password
Redhat

Data transmission using daemon
Pull: rsync [OPTION] [USER @] HOST: SRC [DEST]
Rsync [OPTION] rsync: // [USER @] HOST [: PORT]/SRC... [DEST]
Push: rsync [OPTION] SRC... [USER @] HOST: DEST
Rsync [OPTION] SRC... rsync: // [USER @] HOST [: PORT]/DEST

The client creates a file and pushes it to the server.
[Root @ localhost html] # touch a.txt
[Root @ localhost html] # rsync-avz-P/var/www/html/rbackup@192.168.0.102: www/-- password-file =/etc/rsync. password
Or
[Root @ localhost html] # rsync-avz-P/var/www/html/rsync: // rbackup@192.168.0.102:/www -- password-file =/etc/rsync. password
Building file list...
2 files to consider
./
A.txt
0 100% 0.00kB/s 0:00:00 (xfer #1, to-check = 0/2)

Sent 100 bytes encoded ed 44 bytes 96.00 bytes/sec
Total size is 0 speedup is 0.00

The push is successfully viewed on the server.
[Root @ localhost html] # ls
A.txt index.html index. php

-- Deleted parameter: ensure that the local data is exactly the same as the remote data. If the local data has no remote data, delete it.
Client execution
[Root @ localhost ~] # Mkdir aaa create an empty directory
[Root @ localhost ~] # Rsync-avz-P -- delete/root/aaa/rbackup@192.168.0.102: www/-- password-file =/etc/rsync. password
Building file list...
1 file to consider
Deleting index. php
Deleting index.html
Deleting a.txt
./

Sent 50 bytes encoded ed 22 bytes 13.09 bytes/sec
Total size is 0 speedup is 0.00
Check/var/www/html on the server.
[Root @ localhost ~] # Ll/var/www/html/
Total 0

The configuration file is written in multiple directories on the server.
[Root @ localhost ~] # Vim/etc/rsyncd. conf
Uid = root
Gid = root
Use chroot = no
Max connections = 200
Timeout = 300
Pid file =/var/run/rsyncd. pid
Lock file =/var/run/rsync. lock
Log file =/var/log/rsyncd. log
Ignore errors
Read only = false
List = false
Hosts allow = 192.168.0.0/24
Hosts deny = 0.0.0.0/32
Auth users = rbackup
Secrets file =/etc/rsync. password
[Www]
Path =/www
[Bbs]
Path =/bbs
[Blog]
Path =/blog

Change complete Restart service
[Root @ localhost ~] # Pkill rsync & rsync -- daemon

Rsync + inotify enables triggered automatic synchronization. inotify creates an identical file or a file rsync backup source automatically creates an identical file. inotify deletes a file rsync backup source and automatically deletes the file.

Note: inotify is deployed successfully on the rsync service.

Demo:
Check whether inotify is supported by the current system.
[Root @ localhost ~] # Ls-l/proc/sys/fs/inotify/
Total 0
-Rw-r -- 1 root 0 12-11 max_queued_events # monitoring event queue
-Rw-r -- 1 root 0 12-11 max_user_instances # maximum number of monitored instances
-Rw-r -- 1 root 0 12-11 max_user_watches # maximum number of monitored files per instance

Compile and install inotify
Tar zxf inotify-tools-3.14.tar.gz
Cd inotify-tools-3.14
../Configure -- prefix =/usr/local/inotify-tools-3.14
Make & make install

Script

Host01 = 10.0.0.191 # define an IP address
Src =/data0/www/# define the source directory
Dst = www # target directory
User = rsync_backup # backup user
Rsync_passfile =/etc/rsync. password # password File
Inotify_home =/usr/local/inotify-tools-3.14/# inotify directory
# Judge
If [! -E "$ src"] \
| [! -E "$ {rsync_passfile}"] \
| [! -E "$ {inotify_home}/bin/inotifywait"] \
| [! -E "/usr/bin/rsync"];
Then
Echo "Check File and Folder" # Check whether the above File exists. If it does not exist, an error is reported and the execution is exited.
Exit 9
Fi
$ {Inotify_home}/bin/inotifywait-mrq -- timefmt '% d/% m/% y % H: % m' -- format' % T % w % F'-e close_write, delete, create, attrib $ src \
| While read file # The monitored events delete, create, and attrib are read through read
Do
Cd $ src & rsync-aruz-R -- delete. /-- timeout = 100 $ user @ $ host01: $ dst -- password-file =$ {rsync_passfile}>/dev/null 2> & 1
Done
Exit 0

Put the script in the background for execution
[Root @ localhost ~] # Sh inotify. sh &
[1] 8216
[Root @ localhost ~] # Ps-ef | grep inotify
Root 8216 7503 0 00:00:00 pts/2 sh inotify. sh
Root 8217 8216 0 00:00:00 pts/2/usr/local/inotify-tools-3.14 // bin/inotifywait-mrq -- timefmt % d/% m/% y % H: % M -- format % T % w % f-e close_write, delete, create, attrib/var/www/html
Root 8218 8216 0 00:00:00 pts/2 sh inotify. sh
Root 8220 7503 0 00:00:00 pts/2 grep inotify

Create several files
[Root @ localhost ~] # Cd/var/www/html/
[Root @ localhost html] # touch a B c d
The backup server is synchronized.
[Root @ localhost etc] # cd/var/www/html/
[Root @ localhost html] # ls
A.txt B c d

-------------------------------------- Split line --------------------------------------

Rsync + inotify implement real-time synchronization and backup of Git data

Rsync for file backup Synchronization

Rsync synchronizes two servers

Remote synchronization of Rsync in CentOS 6.5

Use Rsync in Ubuntu Linux for data backup and Synchronization

Linux uses the Rsync client to synchronize directories with the server for backup

Rsync details: click here
Rsync: click here

This article permanently updates the link address: