Rsync + SSH Incremental backup file in FreeBSD system

Rsync + SSH Incremental backup file in FreeBSD system

There are two machines. One is called a server and the other is called a backup machine. How can we copy the server data to the backup machine on a daily or regular basis? The powerful functions of rsync will be used here.

During each transmission, rsync checks the remote file list and transfers only updated files remotely. Therefore, it only takes a long time to run the first time, it will soon be executed later. It also supports ssh transmission, so the security is basically reliable.

How can we implement it?

First, we first define that the server's DN is server.bkjia.com (of course, my family does not have this machine =); the backup machine's DN is called backup.bkjia.com. My account names on both sides are xiaochen.

I. first try.

First, install rsync on both sides. FreeBSD is usually in/usr/ports/net/rsync. If it is not here, find it through whereis rsync. After finding it, go to cd and make install clean.

After the installation is complete, you must execute rehash to update the command list immediately. Then, log on to backup.bkjia.com with xiaochen and write a small Bourne Shell to check whether rsync can operate. Put it in/home/xiaochen/rsync. sh (remember chmod + x ):

[Archive start, excluding this line]

#! /Bin/sh

#! /Bin/sh
RSYNC =/usr/local/bin/rsync
SSH =/usr/bin/ssh
RUSER = xiaochen
RHOST = server.bkjia.com
RPATH =/home/xiaochen/test
LPATH =/mnt/database/fileserver/mirror.bkjia.com
$ RSYNC-e "$ SSH"-avzP -- delete $ RUSER @ $ RHOST: $ RPATH $ LPATH

[End of file, excluding this line]

The variables are described as follows:

* RSYNC is the path where the rsync execution file is located. Each machine is not necessarily the same. You must use whereis rsync to find it.
* SSH is the path where the ssh execution file is located. Of course you should also use whereis ssh to find it.
* RUSER is the name of my account at server.yorkxin.org.
* RHOST is the Domain Name or IP address of the host.
* RPATH is the path to be backed up to the prepare machine on the hosts 』. During rsync, the entire sub-directory will be lost, so you only need to specify the top path.
* LPATH refers to the path on backup.yorkxin.org where I want to place the stock preparation file 』.

A: Keep all attributes of the file.

V: information displayed during execution

Z: compressed files during Synchronization

P: displays the execution process.

The last line is disgusting. In fact, it is the rsync command plus parameters. The usage is

Rsync [parameter] remote user @ remote machine: remote path local path

Parameters are described as follows:

*-E is followed by a string, indicating that we should use different methods to log on to the host machine, such as ssh or rsh.
*-A is archive. To be honest, I don't know what it means. Orz
*-Z requires rsync to be compressed during transmission, saving time
* -- Delete (two dash) requires rsync to "delete files deleted from remote hosts, and delete the same files on the local machine (the stock Preparation Machine 』. This is quite convenient for snapshot.

After saving the disk, run the chmod + x command First (of course, I want to run mkdir in the home directory first). This first execution will take the server (remote) all files in the specified path are copied to the backup machine (Local Machine). We will run it once here. Next we will try again to save time.

2. Password-free SSH access

Since it is a scheduled backup of a job schedule, it cannot be said that you need to manually enter the password each time. Fortunately, ssh provides the key authentication method. Next, I want to implement a pair of authentication keys, which are placed on the Material Preparation Machine and the producer machine, so that when both parties perform rsync backup through ssh, you do not need to enter the password manually.

1. Generate SSH Key Pair

Enter the following command on the backup.bkjia.com (Local Machine) of the stock Preparation Machine: [connect the two lines together]

Ssh-keygen-t dsa-B 1024-f/home/xiaochen/. ssh/

Of course, you can specify the last parameter (-f, specifying the file generation.

The following message is displayed. There is no text to Enter, but you need to press Enter twice:

Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [Press enter directly]
Enter same passphrase again: [Press enter directly]
Your identification has been saved in/home/xiaochen/. ssh/id_dsa.
Your public key has been saved in/home/xiaochen/. ssh/id_dsa.key.pub.
The key fingerprint is:
96: 0a: 51: 79: c2: a5: f3: 6d: 36: 81: fb: 31: 08: 16: f7: b7 xiaochen@backup.bkjia.com

Then you will see two files:

*/Home/xiaochen/. ssh/id_dsa this is the private key (private key, be careful not to flow out)
*/Home/xiaochen/. ssh/id_dsa.pub this is the Public Key (Public Key)

2. Let the host machine accept the Public Key

Then put the public key (. id_dsa.pub) into the target service. You can use FTP to transfer data, but since you are already on the workstation, use ssh scp. The command is as follows: [connect the two lines together]

Scp/home/xiaochen/. ssh/id_dsa.pub xiaochen@server.bkjia.com :~ /

Press Enter to Enter the password so that scp can log on to the host machine (server.bkjia.com) through ssh ).

After that, you can operate on the host machine (server.bkjia.com. I log on to server.bkjia.com as xiaochen. (The following actions are performed on the host machine server.bkjia.com as xiaochen)

1) if no ~ /. Ssh/directory, use mkdir to generate a. ssh/folder, in the home directory, and chmod to 700. (Note that there must be a '.' before the folder name '.')

2) Move the. id_dsa.pub that was just lost with scp ~ /. Ssh:

Mv. id_dsa.pub ~ /. Ssh/authorized_keys

In fact, the backup.bkjia.com can be used for password-free SSH authentication to directly connect to the host machine (server.bkjia.com. However, this kind of connection opportunity "allows all sources and all commands" should be changed now.

Iii. Restrict the source and power of SSH connection for key authentication

The following actions are also performed on server.bkjia.com.

1. Edit ~ /. Ssh/authorized_keys file, before the first word (ssh-dss)

From = "backup.bkjia.com", command = "/home/xiaochen/validate-rsync"

Remember that the word "ssh-dss" must be empty. From is the DN or IP address of backup.bkjia.com, and command is the location where I put the script file to verify the SSH Command.

2. Verify the script file. In this example, the file/home/xiaochen/validate-rsync is used. Remember to save the disk with chmod + x:

[Archive start, excluding this line]

#! /Bin/sh

Case "$ SSH_ORIGINAL_COMMAND" in
*\&*)
Echo "Rejected"
;;
*\(*)
Echo "Rejected"
;;
*\{*)
Echo "Rejected"
;;
*\;*)
Echo "Rejected"
;;
* \ <*)
Echo "Rejected"
;;
*\'*)
Echo "Rejected"
;;
* \ | *)
Echo "Rejected"
;;
Rsync \ -- server *)
$ SSH_ORIGINAL_COMMAND
;;
*)
Echo "Rejected"
;;
Esac

[Archive start, excluding this line]

Note that although ssh online authentication limits available commands, you can log on to the server from the backup machine as long as you obtain the private key, and copy all accessible files, so be careful not to stream out the private key.

Iv. Add key information

Return to backup.bkjia.com. Edit the/home/xiaochen/rsync. sh file in step 1, because you need to add the key information. This is probably changed to the following:

[Archive start, excluding this line]
#! /Bin/sh

RSYNC =/usr/local/bin/rsync
SSH =/usr/bin/ssh
KEY =/home/xiaochen/id_dsa.key # Private KEY Path
# [Add the above line. Of course you need to change it to your private key path]
RUSER = xiaochen
RHOST = 192.168.10.235
RPATH =/home/xiaochen/test
LPATH =/mnt/database/fileserver/mirror.bkjia.com
$ RSYNC-e "$ SSH-I $ KEY"-avzP -- delete $ RUSER @ $ RHOST: $ RPATH $ LPATH

 

# The-I $ KEY parameter must be added to the "Command" in the last line. It is blank after $ SSH.

[End of file, excluding this line]

Run the command once to check whether it is normal.

If everything is normal, you can actually set rsync. the-avz parameter in the last line of sh command is changed to-az. After all,-v is used when we want to "Capture and view progress.

5. Join the work schedule

Run crontab-e in backup.bkjia.com as xiaochen and add

0 5 ***/home/xiaochen/rsync. sh

This is performed at every day. Of course you can change it to another time.

-------------------------------------- Split line --------------------------------------

Rsync + inotify implement real-time synchronization and backup of Git data

Rsync for file backup Synchronization

Rsync synchronizes two servers

Remote synchronization of Rsync in CentOS 6.5

Use Rsync in Ubuntu Linux for data backup and Synchronization

Linux uses the Rsync client to synchronize directories with the server for backup

Rsync details: click here
Rsync: click here

This article permanently updates the link address: