Rsync file synchronization

Summary from the book "Linux O & M path" by Ding yiming

Rsync (Remote Sync) is a magic data image backup software on the UNIX platform. Differential Backup Based on data changes to reduce traffic and improve work efficiency. Rsync uses the tcp873 port.

On the server side:

[[Email protected]/] # mkdir/rsyncfile

[[Email protected]/] # gedit/etc/rsyncd. conf

The added content is:

#/Etc/rsyncd. conf

# Set the name of the server information prompt file and write the prompt information in the file

Motd file =/etc/rsyncd. motd

# Enable the rsync data transmission log function

Transfer logging = Yes

# Set the log file name. You can use the log format parameter to set the log format.

Log File =/var/log/rsyncd. Log

# Set the rsync process number to save the file name

PID file =/var/run/rsyncd. PID

# Set the lock file name

Lock file =/var/run/rsync. Lock

# Set the port number of the server listener. The default value is 873.

Port = 873

# Set the IP address of the NIC interface monitored by the server

Address = 192.168.118.253

# Set the account name or ID used for data transmission. The default value is nobody.

Uid = nobody

Gid = nobody

Use chroot = Yes

# Whether to allow the client to upload data. Set this parameter to read-only.

Read Only = Yes

// Concurrency

Max connections = 10

[Common]

Comment = web content

Path =/rsyncfile

Ignore errors

Auth users = Tom, Jerry

Secrets file =/etc/rsyncd. Secrets

Hosts allow = 192.168.118.0/255.255.255.0

Hosts deny = *

# Whether the module name is displayed when the client requests to display the module list. The default value is true.

List = false

[[Email protected]/] # echo "Tom: 123">/etc/rsyncd. Secrets

[[Email protected]/] # echo "Jerry: 123">/etc/rsyncd. Secrets

It is best to set it to 600 for security purposes

[[Email protected]/] # chmod 600/etc/rsyncd. Secrets

[[Email protected]/] # echo "Welcome to access">/etc/rsyncd. motd

[[Email protected]/] # rsync -- daemon

[[Email protected]/] # echo "/usr/bin/rsync -- daemon">/etc/rc. Local

[[Email protected]/] # iptables-I input-P TCP -- dport 873-J accept

[[Email protected]/] # service iptables save

Iptables: Saving firewall rules to/etc/sysconfig/iptables: [OK]

[[Email protected]/] # Touch rsyncfile/test

[[Email protected]/] #

[[Email protected] ~] # Yum-y install rsync

[[Email protected] ~] # Ls

Anaconda-ks.cfg install. Log install. log. Syslog

[[Email protected] ~] # Mkdir/test

[[Email protected] ~] # Rsync-vzrtopg -- Progress [email protected]: Common/test

Welcome to access

Password:

Refreshing incremental file list

./

Test

0 100% 0.00kb/s 0:00:00 (xfer #1, to-check = 0/2)

Sent 79 bytes received 165 bytes 97.60 Bytes/sec

Total size is 0 speedup is 0.00

It is troublesome to set a password every time.

[[Email protected] ~] # Echo "123"> rsync. Pass

[[Email protected] ~] # Chmod 600 rsync. Pass

[[Email protected] ~] # Rsync-avz -- delete -- password-file = rsync. Pass [email protected]: Common/test

Welcome to access

Refreshing incremental file list

Sent 57 bytes encoded ed 126 bytes 366.00 Bytes/sec

Total size is 0 speedup is 0.00

[[Email protected] ~] #

You can also add a script or even schedule a task. The following shows how to add a script.

[[Email protected]/] # gedit rsync_bak.sh

Script content

#! /Bin/bash

# Backup through rsync

# Data: 2014-6-27

Export Path =/bin:/usr/local/bin

Src = Common

DeST =/Data

Server = 192.168.118.253

User = Jerry

# Password file must not be other-accessible

Passfile =/root/rsync. Pass

# If the Dest direct not found, then create one.

[! -D $ DEST] & mkdir $ dest

[! -E $ passfile] & Exit 2

Rsync-Az -- delete -- password-file = $ passfile $ {user }@$ {server }:: $ SRC $ dest

Executable

[[Email protected] ~] # Sh rsync_bak.sh

Welcome to access

Rsync + inotify for automatic file synchronization

If you only use rsync for data synchronization, you can only meet the requirements of enterprises for real-time data, even if you use scheduled tasks, you can only implement regular data synchronization. In addition, before using rsync for data synchronization, we need to compare all the files and synchronize the differential data. However, our data may only change from 1 kb to 1 TB, to synchronize 1 kb of data without knowing what will change, we need to continuously perform rsync connections to compare the data difference between the client and the server, this is not efficient and requires the inotify tool.

Inotify is an event notification mechanism that enables user-Mode Applications to monitor changes to the file system in real time. However, inotify is only a system function provided by the kernel, to use this function, you also need to install the user-state software.

Go to https://github.com/rvoicilas/inotify-toolsto download the corresponding package.

[[Email protected] inotify-tools-master] # sh autogen. Sh

Libtoolize: Putting auxiliary files in '.'.

Libtoolize: Linking File './ltmain. Sh'

Libtoolize: Putting Macros in ac_config_macro_dir, 'm4 '.

Libtoolize: Linking file 'm4/libtool. m4'

Libtoolize: Linking file 'm4/ltoptions. m4'

Libtoolize: Linking file 'm4/ltsugar. M4'

Libtoolize: Linking file 'm4/ltversion. m4'

Libtoolize: Linking file 'm4/LT ~ Obsolete. M4'

Libtoolize: consider adding '-I M4' to aclocal_amflags in makefile. am.

Configure. AC: 16: Installing './config. Guess'

Configure. AC: 16: Installing './config. sub'

Configure. AC: 6: Installing './install-Sh'

Configure. AC: 6: Installing './missing'

Libinotifytools/src/makefile. AM: 26: docdir was already defined in condition true, which between des condition doxygen_enable...

Configure. AC: 5:... 'docdir' previusly defined here

Libinotifytools/src/makefile. AM: Installing './depcomp'

OK, you can run './configure' now.

[[Email protected] inotify-tools-master] #./configure

[[Email protected] inotify-tools-master] # Make & make install

Start and test:

[[Email protected] inotify-tools-master] # mkdir-P/test

[[Email protected] inotify-tools-master] # echo "hello">/test/foo

[[Email protected] inotify-tools-master] # inotifywait/test/

Setting up watches.

Watches established.

/Test/modify foo

Enter [email protected] ~] on another terminal. # Echo "try">/test/Foo can detect and exit.

You can write a detection script.

[[Email protected] ~] # Cat monitor. Sh

#! /Bin/bash

While inotifywait-e Modify/var/log/messages

Do

If tail-N1/var/log/messages | grep NetworkManager

Then

Echo love

Fi

Done

Example of rsync and inotify:

 

Set in 192.168.118.253:

[[Email protected] ~] # Mkdir-P/var/www/001

[[Email protected] ~] # Chmod 660/var/www/001

[[Email protected] ~] # Chown Nobody. Nobody/var/www/001

[[Email protected] ~] # Gedit/etc/rsyncd. conf

The modification content is as follows:

#/Etc/rsyncd. conf

#

Transfer logging = Yes

Log File =/var/log/rsyncd. Log

PID file =/var/run/rsyncd. PID

Lock file =/var/run/rsync. Lock

Port = 873

Address = 192.168.118.253

Uid = nobody

Gid = nobody

Use chroot = Yes

Read Only = Yes

Max connections = 10

[Web1]

Comment = web content

Path =/var/www/001

Ignore errors

Auth users = Tom, Jerry

Secrets file =/etc/rsyncd. Secrets

Hosts allow = 192.168.118.254

Hosts deny = *

List = false

[[Email protected]/] # echo "Tom: 123">/etc/rsyncd. Secrets

[[Email protected]/] # echo "Jerry: 123">/etc/rsyncd. Secrets

It is best to set it to 600 for security purposes

[[Email protected]/] # chmod 600/etc/rsyncd. Secrets

[[Email protected]/] # echo "Welcome to access">/etc/rsyncd. motd

[[Email protected]/] # rsync -- daemon

[[Email protected]/] # echo "/usr/bin/rsync -- daemon">/etc/rc. Local

[[Email protected]/] # iptables-I input-P TCP -- dport 873-J accept

[[Email protected]/] # service iptables save

Iptables: Saving firewall rules to/etc/sysconfig/iptables: [OK]

Configure the following in the web server 192.168.118.250:

[[Email protected] ~] # Mkdir-P/var/www/002

[[Email protected] ~] # Chmod 660/var/www/002

[[Email protected] ~] # Chown Nobody. Nobody/var/www/002

[[Email protected] ~] # Gedit/etc/rsyncd. conf

The modification content is as follows:

#/Etc/rsyncd. conf

#

Transfer logging = Yes

Log File =/var/log/rsyncd. Log

PID file =/var/run/rsyncd. PID

Lock file =/var/run/rsync. Lock

Port = 873

Address = 192.168.118.250

Uid = nobody

Gid = nobody

Use chroot = Yes

Read Only = Yes

Max connections = 10

[Web1]

Comment = web content

Path =/var/www/002

Ignore errors

Auth users = Tom, Jerry

Secrets file =/etc/rsyncd. Secrets

Hosts allow = 192.168.118.254

Hosts deny = *

List = false

[[Email protected] ~] # Echo "Tom: 123">/etc/rsyncd. Secrets

[[Email protected] ~] # Echo "Jerry: 123">/etc/rsyncd. Secrets

[[Email protected] ~] # Chmod 600/etc/rsyncd. Secrets

[[Email protected] ~] # Echo "Welcome to access">/etc/rsyncd. motd

[[Email protected] ~] # Rsync -- daemon

[[Email protected] ~] # Echo "/usr/bin/rsync -- daemon">/etc/rc. Local

[[Email protected] ~] # Iptables-I input-P TCP -- dport 873-J accept

[[Email protected] ~] # Service iptables save

Iptables: Save the firewall rules to/etc/sysconfig/iptables: [OK]

[[Email protected] ~] #

Configure on 192.168.118.254

[Email protected]:/home/tempal/inotify-tools-master # apt-Get install automake libtool

[Email protected]:/home/tempal/inotify-tools-master # sh autogen. Sh

[Email protected]:/home/tempal/inotify-tools-master #./configure

[Email protected]:/home/tempal/inotify-tools-master # Make & make install

[Email protected]:/home/tempal/inotify-tools-master # echo "123">/root/rsync. Pass

[Email protected]:/home/tempal/inotify-tools-master # Cat/root/rsync. Pass

123

[Email protected]:/home/tempal/inotify-tools-master # chmod 600/root/rsync. Pass

[Email protected]:/home/tempal/inotify-tools-master # CD ..

[Email protected]:/home/tempal # gedit policy_rsync.sh

Add content:

#! /Bin/bash

# This rsync scritpt Based on inotify

# Data: 2014-6-27

Export Path =/bin:/usr/local/bin

Src =/web_data/

Dest1 = web1

Dest2 = web2

Client1 = 192.168.118.253

Client2 = 192.168.118.250

User = Jerry

# Password file must not be other-accessible

Passfile =/root/rsync. Pass

# If the Dest direct not found, then create one.

[! -E $ passfile] & Exit 2

# Wait for change

Inotifywait-mrq -- timefmt '% Y-% m-% d % H: % m' -- format' % T % w % F % E '\

-- Event modify, create, move, delete, attrib $ SRC | while read line

Do

Echo "$ line">/var/log/inotify_web 2> & 1

/Usr/bin/rsync-avz -- delete -- Progress -- password-file = $ passfile $ SRC $ {user} @ $ Client1 :: $ dest1>/var/log/sync_web1 2> & 1

/Usr/bin/rsync-avz -- delete -- Progress -- password-file = $ passfile $ SRC $ {user} @ $ Client2 :: $ dest2>/var/log/sync_web2> & 1

Done &

Run the text and start the system

[Email protected]:/home/tempal # sh notify_rsync.sh

[Email protected]:/home/tempal # echo "/root/policy_rsync.sh"/etc/rc. Local

If the error "/usr/local/bin/inotifywait: Error while loading shared libraries: libinotifytools. so.0" is displayed, the following solutions can be used:

Ln-SV/usr/local/lib/libinotify */usr/lib/

Ln-S/usr/local/lib/libinotifytools. so.0/usr/lib/libinotifytools. so.0

CP/usr/lib/libinotifytools. so.0/usr/local/lib/