Mirroring a Web site with rsync steps _ server Other

For the system administrator, the focus should focus on maintaining the normal operation of the system, to be able to provide services normally, this often involves a data backup problem, in my understanding, 80% of the system administrator is not too concerned about the security of their own servers, but is often very interested in the technology of standby mirroring, But because of the high price of hardware and software in commercial products, free software is often chosen. This is the kind of software that is being introduced here to meet the most demanding backup requirements that are not particularly high.

I. Introduction to the Characteristics

Rsync is a data mirroring Backup tool under Unix-like systems, and--remote sync can be seen from the name of the software. Its characteristics are as follows:

1, can be mirrored to save the entire directory tree and file system.
2, it is easy to maintain the original file permissions, time, soft and hard links and so on.
3, no special permission can be installed.
4, the optimized process, the file transmission efficiency is high.
5, can use RCP, ssh and other means to transfer files, of course, can also be connected through the direct socket.
6, support anonymous transmission.

Ii. Methods of Use

The use of rsync is very simple, I would like to give my own use of examples to illustrate it.

1. System environment

Rsync supports most Unix-like systems, both Linux, Solaris, and BSD are well tested. My system environment is:

Server:freebsd 4.3 ip:192.168.168.52
Client:solaris 8 ip:192.168.168.137
Rsync version 2.4.6 (you can get the latest version from http://rsync.samba.org/rsync/)

2, configure server-side/etc/rsyncd.conf files

bash-2.03# cat/etc/rsyncd.conf

UID = nobody
GID = Nobody
Use chroot = no # chroot not used
Max connections = 4 # Maximum number of connections is 4
PID file =/var/run/rsyncd.pid
Lock file =/var/run/rsync.lock
Log file =/var/log/rsyncd.log # logging Files

[Inburst] # Here is the Certified module name, which needs to be specified on the client side
Path =/home/inburst/python/# A directory that needs to be mirrored
Comment = BACKUP CLIENT is SOLARIS 8 E250
Ignore errors # can ignore some irrelevant IO errors
Read Only = yes # reading only
List = no # does not allow column files
Auth users = inburst # authenticated username, if not, indicates anonymous
Secrets file =/etc/inburst.pas # Certified file name

[web]
Path =/usr/local/apache/htdocs/
Comment = inburst.org Web server

3. Generate a password file on the server side/etc/inburst.pas

bash-2.03# Cat/etc/inburst.pas
Inburst:hack

For security purposes, the properties of a file must be only owner-readable.

4, on the server side will rsync in the form of daemon launch

bash-2.03# rsync--daemon

There are several different ways to start a service at startup, such as:

A, join inetd.conf

Edit/etc/services, add rsync 873/tcp, specify the service port for Rsync is 873
Edit Add/etc/inetd.conf, add rsync stream TCP nowait root/bin/rsync rsync--daemon

B, Join rc.local

In various operating systems, RC file storage location is not the same, you can modify the system to start when Rsync--daemon loaded in.

5, from the client side to test

The following command line in-VZRTOPG V is verbose,z is compressed, R is RECURSIVE,TOPG is to maintain the original attributes of the file, such as the owner, time
The parameters. --progress refers to the display

Detailed progress,--delete means that if the server side deleted the file, then the client also deleted the file, to maintain a true consensus.
In the back of the Inburst@ip,

Inburst is the username in the specified password file, followed by:: Inburst This inburst is the module name, which is the custom in/etc/rsyncd.conf
's name. The last/TMP is a backup

To the local directory name.

In this case, you can also use the e-ssh parameter to establish an encrypted connection. You can use--password-file=/password/path/file to specify the cipher text
So that you can make it in the script

Instead of having to enter the authentication password interactively, it should be noted that this password file permission attribute must be set to only the owner-readable.

bash-2.03# RSYNC-VZRTOPG--progress--delete inburst@192.168.168.52::inburst/tmp/
Password:
Receiving file list ... done
./
1
785 (100%)
1.py
4086 (100%)
2.py
10680 (100%)
A
0 (100%)
Ip
3956 (100%)
./
Wrote 190 bytes read 5499 bytes 758.53 bytes/sec
Total size is 19507 speedup is 3.43

6. Create an update script

If you have more complex work, it can be helpful to take advantage of some common scripting languages. Like what:

bash-2.03# cat/usr/local/bin/rsync.sh

#!/bin/sh
Date= ' Date +%w '

RSYNC-VZRTOPG--progress--delete inburst@192.168.168.52::inburst/home/quack/backup/$DATE--password-file=/etc/ Rsync.pass >

/var/log/rsync. $DATE

7, modify/etc/crontab to do the timing

Like what:

bash-2.03# echo "4 * * 6 root rsync.sh" >>/etc/crontab

Third, FAQ

Q: How to use SSH for rsync without having to enter a password?
A: You can do this by following these steps

1. Set up SSH keys on Server A by Ssh-keygen, do not specify a password, you will see the identity and identity.pub files under ~/.ssh
2. Create subdirectories in the home directory on Server B. SSH
3. Copy A's identity.pub to Server B
4. Add Identity.pub to ~[user B]/.ssh/authorized_keys
5. A user on Server A can then SSH to Server B with User B via the following command
e.g. Ssh-l UserB ServerB
This allows user A on server A to SSH to User B's identity without a password to log on to Server B.

Q: How do I use rsync through a firewall without compromising security?
A: The answers are as follows:

This usually has two kinds of situations, one is the server inside the firewall, one is the server outside the fire wall.
In either case, usually using SSH, it is best to create a new backup user and configure sshd only to allow this user to enter through the RSA authentication method.
If the server is inside the firewall, it is best to qualify the client's IP address and reject all other connections.
If the client is inside the firewall, it is OK to simply allow the firewall to open an SSH outgoing connection to TCP port 22.

Q: Can I back up any files that have been changed or deleted?
A: Of course you can:

You can use such as: Rsync-other-options-backupdir =./backup-2000-2-13 ... Such a command to implement.
So if the source file:/path/to/some/file.c changed, the old file will be moved to./BACKUP-2000-2-13/PATH/TO/SOME/FILE.C,
This catalogue here needs its own

Built by hand.

Q: What ports do I need to open on the firewall to accommodate rsync?
A: Depending on the situation

Rsync can transfer files directly through a TCP connection on port 873, or through 22-port SSH for file delivery, but you can also change its port via the following command:

Rsync--port 8730 otherhost::
Or
Rsync-e ' Ssh-p 2002 ' Otherhost:

Q: How can I just copy the directory structure through rsync, ignoring the files?
A:rsync-av--include ' * * *--exclude ' * ' Source-dir dest-dir

Q: Why do I always have the "Read-only file system" error?
A: see if you forgot to set "Read Only = no"

Q: Why do I appear ' @ERROR: Invalid gid ' error?
A:rsync Use the default is to run with Uid=nobody;gid=nobody, if your system does not exist nobody group, there will be such a mistake, you can try the GID =

Nogroup or anything.

Q: Binding port 873 What's wrong with failure?
A: If you are not running this daemon as root, this error can occur because the port below port 1024 is privileged. You can use the--port parameter to change.

Q: Why did I fail the certification?
A: From your command line, it appears:

You use the following:
> bash$ rsync-a 144.16.251.213::test Test
> Password:
> @ERROR: Auth failed on module test
>
> I dont understand this. Can somebody explain as to the acomplish this.
> All suggestions are welcome.

There should be no problem with your username landing, try rsync-a max@144.16.251.213::test test

Iv. some scripts to learn from

These scripts here are examples of rsync sites:

1, every seven days the data to the central server to do incremental backup

#!/bin/sh

# This script does personal backups to a rsync backup server. You'll end up
# with a 7 day rotating incremental backup. The Incrementals'll go
# into subdirectories named after the day of the week, and the current
# full backup goes into a directory called ' current '
# tridge@linuxcare.com

# Directory to Backup
bdir=/home/$USER

# excludes File-this contains a wildcard pattern/line of files to exclude
excludes= $HOME/cron/excludes

# The name of the backup machine
Bserver=owl

# Your password on the backup server
Export Rsync_password=xxxxxx


########################################################################

Backupdir= ' Date +%a '
opts= "--force--ignore-errors--delete-excluded--exclude-from= $EXCLUDES
--delete--backup--backup-dir=/$BACKUPDIR-a "

Export path= $PATH:/bin:/usr/bin:/usr/local/bin

# The following line clears weeks incremental directory
[-D $HOME/emptydir] | | mkdir $HOME/emptydir
rsync--delete-a $HOME/emptydir/$BSERVER:: $USER/$BACKUPDIR/
RmDir $HOME/emptydir

# now the actual transfer
Rsync $OPTS $BDIR $BSERVER:: $USER/current

2, Backup to a free hard drive

#!/bin/sh

Export Path=/usr/local/bin:/usr/bin:/bin

list= "Rootfs usr data data2"

For d in $LIST; Todo
mount/backup/$d
Rsync-ax--exclude fstab--delete/$d//backup/$d/
umount/backup/$d
Done

day= ' Date ' +%a '

Rsync-a--delete/usr/local/apache/data2/backups/$DAY
Rsync-a--delete/data/solid/data2/backups/$DAY

3, mirror the CVS Tree of vger.rutgers.edu

#!/bin/bash

cd/var/www/cvs/vger/
Path=/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin

run= ' LPs x | grep rsync | Grep-v grep | Wc-l '
If ["$RUN"-GT 0]; Then
Echo already running
Exit 1
Fi

Rsync-az Vger.rutgers.edu::cvs/cvsroot/changelog $HOME/changelog

sum1= ' Sum $HOME/changelog '
Sum2= ' Sum/var/www/cvs/vger/cvsroot/changelog '

If ["$sum 1" = "$sum 2"]; Then
Echo Nothing Todo
Exit 0
Fi

Rsync-az--delete--force vger.rutgers.edu::cvs//var/www/cvs/vger/
Exit 0

4, using Find a clever way

RSYNC-AVR remote: ' find/home-name ' *. [CH] "'/tmp/

You can use this method to list the files you need to back up--a method that seems less likely to be used by people.

V. reference materials:

1, http://rsync.samba.org/
2, rsync examples
3. Rsync FAQ