Rsync+inotify for full-network Automation data backup-technology flow Ken

1.rsync Introduction

"Rsync is a data-mirroring backup tool for Linux systems. Use the Quick Incremental Backup tool Remote Sync can be synchronized remotely, support local replication, or synchronize with other SSH, Rsync hosts

2.rsyncfeatures and features of

1. can make backup of various resources of server (can boast file system)

2. Linux-rsync,windows-cwrsync

3. can do full-scale backup, can also do incremental backup

4. in the backup, you can exclude some specific files do not do backup

5. encrypted transmission can be implemented in conjunction with SSH

6. Rsync supports working in the background mode (daemon mode)

7. Rsync can be combined with inotify/sersync for automatic real-time backup

8. Rsync can do speed limit in the process of transferring files

3. The working mode of rsync

1. command-line mode (shell mode)

similar to CP,dd command for backup File Replication (Backup)

2. Remote mode (remote shell mode)

Remote transfer of data using SSH, similar to SCP

3. List mode

similar to executing the LS command, just for listing the file contents list (not doing a copy operation)

4. background mode (daemon mode)

Rsync is working in the background.

4.rsync for automatic backup

Rsync Real-time automatic backup

combined with Inotify/sersync

Inotify/sersync: Whether files in the monitoring files and directories are changed (find new files)

Rsync: Back up the files that were found in the past

usage of rsync

1. Local Use (cp)

2. remote Use (SCP)

3. Daemon (socket)

5. rsync Detailed Options

option -p: Keep file Permissions property unchanged-V: Display execution process information -r: Directory to do recursion -A: archive (contains R)-z: Compress -  L: Just transfer the soft link itself -L: The original file that the soft connection points to-B: When backing up the file, if the backup file already exists, the old file under the target location is renamed, and then a new backup is generated --suffix=  XXX        Specifies the suffix name of the old backup file --backup-dir=xxxx Specifies where to move the   old backup file under -e: Encrypted transmission in conjunction with SSH "  ssh-p" combination:-avz

6. rsync configuration file parameters detailed

module: is actually a directory, this module is used to save the client passed over the file    parameters:        pid file: Specify the location of the PID files of rsync        UID: Specify the User ID        to run the rsync process GID: Specifies the user group ID that is running the rsync process Port        : Specifies the ports that rsync listens on (default port 873)        Path: Specifies the location of the directory for the module        use chroot: Specifies to lock the user in        the home directory Max connections: Maximum number of connections (specifies how many clients are transferring files at the same time)        log file: Specify the location of the log files log format        : Specify the log        formats lock file: Specifies the location of the lock file for the rsync process        Read Only: Specifies whether the module is read-only state        write only: Specifies whether the module is write        -only List: Specifies whether the user can view the file lists under the path corresponding to the module        exclude: Specify exclude files        from being transferred from : Read a file to get not what files are transmitted        Auth Users: Specifies the anonymous user name to use to connect to this module        Secrets file: Specifies the save virtual user and password file hosts        allow: Specifies the host hosts that can do file transfers        Deny: Blacklist        timeout: Specify the time-out for the client

7.inotify Introduction

INotify: finds files or new files that have been sent, and can continuously detect whether a file in a directory has changed.

software Inotify-tools has two components

Inotifywait: Detects events that occur in a file under a directory

Inotifywatch: Count the number of events that are triggered

8. options for inotifywait

    -R: Recursive    directory -q: Only print monitored event information     -M: Always in the monitoring state [combined use-MQR]    --  Excludei: Exclude file or directory    (case insensitive)--exclude: Exclude file or directory (case sensitive)    --format: Specify output format         % W: Shows the path of the event that triggered the monitoring. [%w%f combination can be used to get the        file name] %F: Displays the file name        of the event that triggered the monitor %E: Displays the event         that was triggered%T:    shows the event's trigger time --  Timefmat: Specify the format    of the output time- e: Specify the events to monitor

9.inotify Monitored events

Access file or directory contents were read modify file or directory contents were WR Itten attrib file or directory attributes changed close_write file or directory closed, after Being openedinchwriteable mode close_nowrite file or directory closed, after being openedinchread-Only mode close file or directory closed, regardless of read/Write mode open file or directory opened moved_to file or directory moved to watched Direc Tory Moved_from file or directory moved fromwatched directory move file or directory moved to or fromwatched directory create file or directory created within watched directory delete fi              Le or directory deleted within watched directory Delete_self file or directory was deleted unmount File system containing file or directory unmounted

10.rsync+inotify full Network Automation backup actual case 1. Environmental preparedness

centos7.5

Rsync Service Side IP:172.20.10.7/28

INotify Service End IP:172.20.10.8/28

2. Turn off security services

[Email protected] ~0~]# iptables-f

3. Configure the Rsync service side

Create a shared directory

[Email protected] ~]# Mkdir/ken

Create a user to run the rsync process

333 KenKen

Modifying the owner and array

[Email protected] ~]# chown-r Kenken.kenken/ken

View rsync files,/etc/rsyncd.conf, this file does not exist by default and needs to be written by hand.

[Email protected] ~]# rpm-/etc/rsyncd.conf/etc/sysconfig/rsyncd/usr/bin/rsync ... 

Create an rsync configuration file

[Email protected] ~]# vim/etc/rsyncd.conf pid File=/var/Lock/subsys/PidfileLockfile=/var/Lock/subsys/Rsynclog File=/var/log/Rsyncuid=333GID=333Timeout= -Max Connections=199[Ken]path=/kenlist=Yesuse chroot=Yesread only=Noauth Users=user1secrets File=/etc/rsyncd.pwdhosts Allow=172.20.10.8/ -

Create a virtual user file

" user1:123 ">>/etc/rsyncd.pwd

Modify permission to 600

600

Start the Rsync service

[[email protected] ~]# rsync--873LISTEN     0      5            *:873                       *:*                  LISTEN     0      5           ::873                     :::*                  

4. Configure the INotify service side

Download inotify need to configure the Epel warehouse, copy the following code into your Yum config file can be

[Ken]name=kenenabled=1gpgcheck=0baseurl=https://  mirrors.aliyun.com/epel/7server/x86_64/

Download inotify

[email protected] ~]# Yum install rsync inotify-tools-y

Create a directory where you want to back up data

[Email protected] ~]# Mkdir/kenken

Create a file that holds the rsync virtual user and password and change the permissions to 600

" 123 " >>/etc/ /etc/rsync.pwd

Create automated monitoring Backup scripts

[[Email protected] ~]# vim authbak.sh#!/bin/Bashprog="inotifywait"Events="Create,delete,close_write"opt="-mrq"dir="/kenken"Remote_host="172.20.10.7"Anon_user="User1"Mod_name="Ken"Pwd_file="/etc/rsync.pwd"$prog $opt-E $events--format"%w%f"$dir | whileRead Line Dorsync-rz--delete $dir [email protected] $remote _host:: $mod _name--password-file=$pwd _filedone

Client Run script

[Email protected] ~]# nohup bash authbak.sh &

5. Testing on the client

Client-side Create file test

[[email protected] kenken]# Touch {1.. Ten }.txt[[email protected] kenken]# lsten. txt  1.txt  2. txt  3  . txt 4. txt 5. txt 6. txt      7.  8. txt  9. txt

See if you have the same files on the rsync server side

[[email protected] ken]# ls   1. txt  2. txt  3. txt  4. txt  5. TXT  6. txt  7. txt  8. txt  9. txt

Test success!

Client Delete file test

1 4
[[email protected] kenken]# ls
10.txt  2.txt  3.txt  5.txt  6.txt  7.txt  8.txt  9.txt

See if the file has also been deleted on the rsync server side

[[email protected] kenken]# ls   2. txt  3. txt  5. txt  6. txt  7. TXT  8. txt  9. txt

Test success!

At this point we can realize the whole network automation, no differential data backup.

Rsync+inotify for full-network Automation data backup-technology flow Ken