Centos 5.8 Builds rsync server

For any internet company, the backup of the data is important.

Original blog from: www.51niux.com bo: Busy Firewood less

Our production environment is like this, we have hundreds of servers, and then in the cloud host room has several physical servers, the size of the space is 10T, the cloud host of course, the general hard disk space is small not suitable for backup server, this time I played the physical machine attention, of course, can only play their ideas.

In the case of data transfer to the physical drive, we list our data transfer policy:

First Scenario: SCP programme

Personally, if it is a separate file, such as a file like the full archive of the transfer database, each server to be transferred sends its own public key to the physical machine.

But we are hundreds of servers, in addition to every day to back up the database of compressed files, there are log logs, as well as the configuration file directory, as well as the daily update of the Code program, every day the program to crawl the various resources on the net, a think is very large, obviously a lot of things can not be backed up completely.

Second Scenario: NFS Scenarios

Let's make a larger storage disk into shared storage, NFS mounted on each server. Some people will say, the server must have write permission Ah, then your other server put the backup on the NFS server, will not be read and write to other servers. We can NFS server with the server host name in a bunch of sub-directories to mount the past, for example, you are 10.1.15.96 server, I will give you in my NFS server total directory/data below the creation of a 10.1.15.96 sub-directory mount the past, so the problem is resolved. But the problem comes again, you want to incremental backup or need rsync, and if the server can also be divided into directories, if there are more than 100 servers, you do not complete 100 sub-directories.

A third scenario: Rsync server

Because of our real environment, we have hundreds of servers, we directly build rsync server, take the database compression packet full-volume transmission, Bin-log binary files such as incremental backup (for MySQL can also take the form of master-slave replication to Do data backup), other configuration directory incremental backup form.

Now that you feel like you're taking this approach with layers of exclusion, you need to plan well:

1. rsync server Open the firewall, only allow the springboard can be accessed through 22 ports, only a network segment open 873 port, set to prohibit root remote access, only allow key login. This allows us to securely harden the rsync server from an external level.

2. rsync server in the configuration of the time to set up a number of directory modules corresponding to each server and to limit, only allow a server to the rsync transmission of this directory, this time may ask you do not have to set up a number of sub-directories, in fact, this should be with the actual demand, Many servers above the data is not how to change, many servers above the data may not be important as long as there are a few fully prepared, only those key data for a single directory module settings, you can do some of the shared directory module, set up to allow those IP access.

3. Data multi-point backup, if we have a server is compromised, such as 10.1.15.99 this server, every day to perform rsync to put the data incrementally into their own separate directory, when this server was breached executed a rsync-avz-delete/tmp/, The things you have worked hard to back up will be clear to you directly. I feel like you can use the monitoring with rsync, for example, another storage nightly increment from the rsync server to copy data, write a script, set a threshold, to determine the size of the directory to be copied, if less than a threshold, do not do rsync-delete operation, In this way our server has two additional storage backup data, in addition to its own backup data. Of course, the details can also be adjusted according to the actual conditions.

Say more, what to do or according to the actual situation, the specific circumstances to consider what plan.

Here are the Rsync server and the client building process:

Environment: Operating system Centos 5.8 rsync server: 10.1.15.96 rsync client: 10.1.15.97 and 10.1.15.238

1). Build rsync Server

The first step: check whether the machine has rsync service, the general installation system comes with

[Email protected] ~]# rpm-qa|grep rsync #如果没有请yum下载.
rsync-3.0.6-4.el5_7.1

Step two, generate the relevant files

[Email protected] rsyncd]# MKDIR/ETC/RSYNCD #用来存放密码文件的目录以及密码用的目录

[Email protected] rsyncd]# vi/etc/rsyncd.conf
Port = 873 #设置监听端口
UID = root #制定传输文件的用户名
GID = root #制定传输文件的密码
Use chroot = Ture #若为 true, rsync chroot to the directory specified by the path parameter before transferring the file. The reason for this is to implement additional security, but the disadvantage is that root permissions are required, and you cannot back up the directory files that point to a symbolic connection that is external to the path.
Read Only = no #可以写, yes is read only
Hosts allow = 10.124.0.0/24 #允许那些ip网段访问
Hosts deny = * #拒绝剩下的所有ip
PID file =/var/run/rsyncd.pid #指定rsyncd服务器的pid文件
Secrets file =/etc/rsyncd/rsyncd.passwd #指定rsyncd. passwd Password files
Log file =/etc/rsyncd/rsync.log #制定rsync. Log transfer log, it is recommended to generate new file timing cleanup every day
Transfer logging = yes #使 the rsync server logs the transfer operation to the transfer log file, you are false
Log format =%t%a%m%f%b #定义指定传输日志文件的字段.
Timeout = #定义超时时间300秒
[10.1.15.238]
Path =/data2/10.1.15.238
List = no #不列出文件列表
Ignore Errors #忽略io错误
Auth users = admin
Hosts allow = 10.1.15.238
Hosts deny = *
Secrets file =/etc/rsyncd/10.1.15.238/rsyncd.passwd

[10.1.15.97]
Path =/data2/10.1.15.97
List = no
Ignore errors
Auth users = admin
Hosts allow = 10.1.15.97
Hosts deny = *
Secrets file =/etc/rsyncd/10.1.15.97/rsyncd.passwd

Step three: Generate directories and files according to the directory of the configuration files

[Email protected] rsyncd]# ls-l/data2/
Total 24
Drwxr-xr-x 5 root root 4096 Mar 25 17:51 10.124.19.97
Drwxr-xr-x 3 root root 4096 Mar 25 17:48 10.124.32.238

[Email protected] rsyncd]# ls-l/ETC/RSYNCD
Total 24512
Drwxr-xr-x 2 root root 4096 Mar 25 17:43 10.1.15.97
Drwxr-xr-x 2 root root 4096 Mar 25 17:19 10.1.15.228

[[email protected] rsyncd]# cat/etc/rsyncd/10.1.15.97/rsyncd.passwd #文件内容是用户名: Password
admin:123456 #密码要复杂点

chmod rsyncd.passwd #必须600授权不然成功不了

Fourth step, start the rsync service

[Email protected] rsyncd]#/usr/bin/rsync--daemon &

Here is the spatial structure of my storage:

Fifth Step: Add firewall rules

[Email protected] rsyncd]# Cat/etc/sysconfig/iptables|grep 873
-A rh-firewall-1-input-s 10.1.15.0/24-p tcp-m tcp--dport 873-j ACCEPT
[Email protected] rsyncd]#/etc/init.d/iptables restart

2). The configuration of the Rsync client

[email protected]. ssh]# cat/etc/rsyncd.passwd #生成只有密码的配置文件
123456

[Email protected] backup]# chmod 600/etc/rsyncd.passwd

Client firewalls do not add rules:

[email protected]. ssh]# Cat/etc/sysconfig/iptables|grep 873
[email protected]. ssh]#

Rsync-avz --delete /data/backup [email protected]::10.1.15.97--password-file=/etc/rsyncd.passwd #真实IP地址能给大家看哈不好意思
#这表示成功了哈, there are no error errors.

So let's try transferring files to another module. Try it:

[email protected]. ssh]# rsync-avz/data/backup [email protected]::10.1.15.238--password-file=/etc/rsyncd.passwd #这是 Another module that we set up
@ERROR: Unknown module ' 10.1.15.238 ' #报错失败了
Rsync Error:error starting Client-server Protocol (code 5) at MAIN.C (1530) [sender=3.0.6]

Let's switch to 10.1.15.238 and try to see if it's a server-side problem or we've set it up successfully:

[[email protected] @uusee ~]$ ssh 10.1.15.238
Last login:wed Mar 17:57:15 from

[Email protected]_238 backup]# rsync-avz xhrdb_2015-03-26-00.sql.gz [email protected]::10.1.15.238--password-file=/ etc/rsyncd.passwd

Sending incremental file list
Xhrdb_2015-03-26-00.sql.gz
Sent 331654361 bytes received bytes 3267530.92 bytes/sec #这状态是成功的

Last step: Add a Scheduled task

[Email protected]_218 backup]# cat/var/spool/cron/root |grep rsync
#mysql_rsync
0 1 * * * rsync-avz/data/backup/[email protected]::10.1.15.238/mysql_backup/--password-file=/etc/rsyncd.passwd

The rest of the line in accordance with their own ideas to deploy the environment, it is a bit messy ha!

Original blog from: www.51niux.com bo: Busy Firewood less

Link: http://www.cnblogs.com/mchina/p/2829944.html is very detailed.

This article from "Technology in the progress of sharing" blog, declined reprint!

Build an rsync server for Centos 5.8