Back up images of websites using rsync (unreliable)

For system administrators, the focus of their work should be on maintaining the normal operation of the system and providing services normally. Here, a data backup problem is often involved.

In this case, 80% of system administrators are not very concerned about the security of their servers, but they are often quite interested in the backup image technology. However, due to the high hardware and software prices of commercial products

So we often choose free software. The Rsync mentioned here is such a software that can meet the vast majority of backup requirements that are not particularly high.

I. Features

Rsync is a data image backup tool in Unix-like systems. It can be seen from the software name-Remote Sync. Its features are as follows:

1. images can be used to save the entire directory tree and file system.
2. It is easy to maintain the permissions, time, and soft links of the original file.
3. installation without special permissions.
4. optimized process, high file transmission efficiency.
5. You can use RCP, ssh, and other methods to transmit files. Of course, you can also use a direct socket connection.
6. Anonymous transmission is supported.

Ii. Usage

Rsync is easy to use. I will illustrate it with my own example.

1. System Environment

Rsync supports most UNIX-like systems, including Linux, Solaris, and BSD. My system environment is:

Server: FreeBSD 4.3 IP: 192.168.168.52
Client: Solaris 8 IP: 192.168.168.20.
Rsync version 2.4.6 (available in the latest version from http://rsync.samba.org/rsync/)

2. Configure the/etc/rsyncd. conf file on the server.

Bash-2.03 # Cat/etc/rsyncd. conf

Uid = nobody
Gid = nobody
Use chroot = No # Do not use chroot
Max connections = 4 # the maximum number of connections is 4
PID file =/var/run/rsyncd. PID
Lock file =/var/run/rsync. Lock
Log File =/var/log/rsyncd. log # log record file

[Inburst] # The Authentication Module name must be specified on the client.
Path =/home/inburst/Python/# directory for Image Creation
Comment = backup client is Solaris 8 e250
Ignore errors # ignore unrelated Io errors
Read Only = yes # Read-Only
List = No # column Files Not Allowed
Auth users = inburst # The authenticated user name. If this row is not used, it indicates that it is anonymous.
Secrets file =/etc/inburst. Pas # authentication file name

[Web]
Path =/usr/local/Apache/htdocs/
Comment = inburst.org Web Server

3. Generate a password file on the server/etc/inburst. Pas

Bash-2.03 # Cat/etc/inburst. Pas
Inburst: hack

For security purposes, the attributes of a file must be readable only by the owner.

4. Start rsync on the server as a daemon

Bash-2.03 # rsync -- daemon

There are several different methods to get the service up at startup, such:

A. Add inetd. conf

Edit/etc/services, add rsync 873/tcp, and set the service port of rsync to 873.
Add/etc/inetd. conf and rsync stream tcp Nowait root/bin/rsync -- daemon

B. Add RC. Local.

In various operating systems, RC files are stored in different locations and can be modified so that rsync-daemon is loaded when the system starts.

5. test from the client

In the following command line-In vzrtopg, V is verbose, z is compression, R is recursive, and topg is a parameter that maintains the original file attributes, such as the owner and time. -- Progress indicates display

For detailed progress, -- delete indicates that if the server deletes the file, the client also deletes the file accordingly to ensure true consistency. In inburst @ IP,

Inburst is the user name in the specified password file, followed by: inburst this inburst is the module name, that is, the name customized in/etc/rsyncd. conf. The last/tmp is backup.

To the local directory name.

You can also use the-e SSH parameter to establish an encrypted connection. You can use -- password-file =/password/path/file to specify the password file.

You do not need to enter the authentication password interactively. Note that the permission attribute of this password file must be set to only the owner and readable.

Bash-2.03 # rsync-vzrtopg -- Progress -- delete inburst@192.168.168.52: inburst/tmp/
Password:
Processing ing file list... Done
./
1
785 (100%)
1. py
4086 (100%)
2. py
10680 (100%)
A
0 (100%)
IP
3956 (100%)
./
Wrote 190 bytes read 5499 bytes 758.53 Bytes/sec
Total size is 19507 speedup is 3.43

6. Create an update script

Some common scripting languages can be helpful for complex work. For example:

Bash-2.03 # Cat/usr/local/bin/rsync. Sh

#! /Bin/sh
Date = 'date + % W'

Rsync-vzrtopg -- Progress -- delete inburst@192.168.168.52: inburst/home/quack/backup/$ date -- password-file =/etc/rsync. Pass>

/Var/log/rsync. $ date

7. Modify/etc/crontab to make the timer

For example:

Bash-2.03 # echo "15 4 ** 6 root rsync. Sh">/etc/crontab

Iii. FAQs

Q: How can I perform rsync through SSH without entering a password?
A: follow these steps:

1. Use SSH-keygen to create an SSH keys on server a. do not specify a password ~ /. Ssh: the identity and identity. Pub files are displayed.
2. Create a subdirectory. SSH in the home directory on server B.
3. Copy identity. Pub of a to server B.
4. Add identity. Pub ~ [User B]/. Ssh/authorized_keys
5. So user a on server a can use the following command to SSH user B to server B.
E.g. Ssh-l userb serverb
In this way, user a on server a can log on to server B as user B without a password.

Q: How can I use rsync through the firewall without compromising security?
A: The answer is as follows:

There are two common cases: one is that the server is inside the firewall and the other is outside the firewall.
In either case, SSH is usually used. In this case, it is best to create a backup user and configure sshd to only allow this user to access through RSA Authentication.
If the server is in the firewall, it is best to limit the IP address of the client and reject all other connections.
If the client is in the firewall, you can simply allow the firewall to open the SSH outbound connection on TCP port 22.

Q: Can I back up the changed or deleted files?
A: Of course:

You can use commands such as rsync-other-options-backupdir =./backup-2000-2-13.
In this case, if the source file is/path/to/Some/file. c changed, so the old file will be moved. /backup-2000-2-13/path/to/Some/file. c. You need to use this directory.

Manually created

Q: What ports do I need to open on the firewall to adapt to rsync?
A: depends on the situation.

Rsync can directly transfer files through TCP connection on port 873, or through SSH on port 22, but you can also change the port through the following command:

Rsync -- Port 8730 otherhost ::
Or
Rsync-e 'ssh-P 2002 'otherhost:

Q: How can I copy only the directory structure through rsync and ignore the file?
A: rsync-Av -- include '*/' -- exclude '* 'source-Dir DEST-Dir

Q: Why do I always see the "read-only file system" error?
A: Check if you forget to set "Read Only = No ".

Q: Why do I encounter the '@ error: Invalid gid' error?
A: During rsync, uid = nobody is used by default; gid = nobody is used for running. If your system does not have a nobody group, this error will occur. You can try gid =

Nogroup or other

Q: Why does port 873 fail to be bound?
A: If you do not run the daemon with the root permission, this error will occur because the ports below port 1024 are privileged ports. You can use the -- port parameter to change the value.

Q: Why does my authentication fail?
A: From the Perspective of your command line:

You are using:
> Bash $ rsync-A 144.16.251.213: Test test
> Password:
> @ Error: AUTH failed on module test
>
> I dont understand this. Can somebody explain as to how to acomplish this.
> All suggestions are welcome.

There should be no issues caused by login with your username, try rsync-A max@144.16.251.213: Test test

4. Some useful scripts

These scripts are examples on the rsync Website:

1. Incremental backup of data to the central server every seven days

#! /Bin/sh

# This script does personal backups to a rsync backup server. You will end up
# With a 7-day rotating Incremental backup. The incrementals will go
# Into subdirectories named after the day of the week, and the current
# Full backup goes into a directory called "current"
# Tridge@linuxcare.com

# Directory to backup
Bdir =/home/$ user

# Excludes file-This contains a wildcard pattern per line of files to exclude
Excludes = $ home/cron/Excludes

# The name of the Backup Machine
Bserver = Owl

# Your password on the backup server
Export rsync_password = xxxxxx

######################################## ################################

Backupdir = 'date + %'
Opts = "-- force -- ignore-errors -- delete-excluded -- exclude-from = $ excludes
-- Delete -- backup-Dir =/$ backupdir-"

Export Path = $ path:/bin:/usr/local/bin

# The following line clears the last weeks incremental directory
[-D $ home/emptydir] | mkdir $ home/emptydir
Rsync -- delete-A $ home/emptydir/$ bserver: $ user/$ backupdir/
Rmdir $ home/emptydir

# Now the actual transfer
Rsync $ opts $ bdir $ bserver: $ user/Current

2. Back up data to an idle Hard Disk

#! /Bin/sh

Export Path =/usr/local/bin:/usr/bin:/bin

List = "rootfs USR data data2"

For D in $ list; do
Mount/backup/$ d
Rsync-ax -- exclude fstab -- delete/$ D // backup/$ D/
Umount/backup/$ d
Done

Day = 'date "+ % "'

Rsync-A -- delete/usr/local/Apache/data2/backups/$ day
Rsync-A -- delete/data/solid/data2/backups/$ day

3. image the CVS tree of vger.rutgers.edu

#! /Bin/bash

CD/var/www/CVS/vger/
Path =/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin

Run = 'lps X | grep rsync | grep-V grep | WC-l'
If ["$ run"-GT 0]; then
Echo already running
Exit 1
Fi

Rsync-Az vger.rutgers.edu: CVS/cvsroot/changelog $ home/changelog

Sum1 = 'sum $ home/changelog'
Sum2 = 'sum/var/www/CVS/vger/cvsroot/changelog'

If ["$ sum1" = "$ sum2"]; then
Echo nothing to do
Exit 0
Fi

Rsync-Az -- delete -- force vger.rutgers.edu: CVs // var/www/CVS/vger/
Exit 0

4. A clever way to use find

Rsync-AVR remote: ''find/home-name "*. [CH]" ''/tmp/

You can use this method to list the files to be backed up. This method seems to be rarely used.

V. References:

1. http://rsync.samba.org/
2. Rsync examples
3. Rsync FAQ