Three SVN backup methods in Linux

(The example in this article is implemented based on FreeBSD/Linux, and you need to modify it on your own in the Windows environment)
An important mission of configuration management is to ensure data security and prevent the catastrophic consequences of the failure to recover data due to disk damage or misoperations on the server. Therefore, it is very important to develop a complete backup policy.

Generally, the backup policy should specify the following parts: Backup Frequency, backup mode, backup storage location, backup owner, disaster recovery inspection measures and provisions.

The backup frequency, storage location, and other content can be customized according to your actual situation. This article focuses on the backup method.

SVN backup generally uses three methods: 1) svnadmin dump 2) svnadmin hotcopy 3) svnsync.

Note: SVN backup should not adopt the normal file copy method (unless you pause the database during Backup), such as the Copy command and rsync command.
I used the rsync command for incremental and full backup. During the quarterly backup check and audit, I found that most of the databases backed up are unavailable. Therefore, it is best to use the functions provided by SVN for backup.

Advantages and disadvantages:
====================
The first svnadmin dump is an officially recommended backup method. It has the advantage of being flexible, supports full backup or Incremental backup, and provides a version recovery mechanism.
Disadvantages: if the version is relatively large, for example, if the number of versions increases to tens of thousands or hundreds of thousands, the dump process will be very slow; backup time consumption and recovery time consumption will be more time consuming; it is not conducive to rapid disaster recovery.
We recommend that you use this backup mode when the number of versions is small.
The original design purpose of svnadmin hotcopy is not estimated to be used for backup. It can only be used for full copy, but not Incremental backup;
The advantage is that the backup process is fast and disaster recovery is fast. If the svn service has been set up on the backup machine and you do not need to recover it, you only need to perform simple configuration to switch to the backup database.
Disadvantage: it is relatively hard disk-consuming and requires large hard disk support (the backup machine has 1 TB space ).
The third type of svnsync is actually to create two image libraries. When one fails, it can be quickly switched to another. However, this function must be supported only in svn1.4 or later versions.
Advantage: When two image libraries are created, the dual-host real-time backup function is enabled;
The disadvantage is that, when used as two image libraries, we cannot "completely discard the changes we made today and restore them to the last night". When we backup them daily as a normal backup mechanism, the operation is more difficult than the first two methods.

The following describes the three backup methods:
====================

1. svnadmin dump Backup Tool
------------------------
This is a backup method officially recommended by subversion.

1) define a backup policy:
Backup Frequency: Full backup is performed every Saturday and Incremental backup is performed from Sunday to Friday.
Backup location: backup storage path to/home/backup/SVN/
Backup name: Full backup file name: weekly_fully_backup.yymmdd, Incremental Backup File Name: daily-incremental-backup.yymmdd
Backup time: Start at every night
Backup check: svnadmin load recovery test is performed at the end of each month.
2) create a full backup script:
In ~ , If there are multiple databases, change them accordingly ):

#! /Usr/bin/perl-W
My $ svn_repos = "/home/SVN/repos/project1 ";
My $ backup_dir = "/home/backup/SVN /";
My $ next_backup_file = "weekly_fully_backup.". 'date + % Y % m % d ';

$ Youngest = 'svnlook youngest $ svn_repos ';
Chomp $ youngest;

Print "backing up to revision $ youngest/N ";
My $ svnadmin_cmd = "svnadmin dump -- Revision 0: $ youngest $ svn_repos> $ backup_dir/$ next_backup_file ";
'$ Svnadmin_cmd ';
Open (log, "> $ backup_dir/last_backed_up"); # record the Backup Version Number
Print log $ youngest;
Close log;
# If you want to save space, execute the following compression script.
Print "compressing dump file.../N ";
Print 'gzip-G $ backup_dir/$ next_backup_file ';

3) create an incremental backup script:
Perform Incremental backup on the basis of full backup ~ /To create a Perl script file named daily_backup.pl. The Code is as follows:

#! /Usr/bin/perl-W
My $ svn_repos = "/home/SVN/repos/project1 ";
My $ backup_dir = "/home/backup/SVN /";
My $ next_backup_file = "daily_incremental_backup.". 'date + % Y % m % d ';

Open (in, "$ backup_dir/last_backed_up ");
$ Previous_youngest = <in>;
Chomp $ previus_youngest;
Close in;

$ Youngest = 'svnlook youngest $ svn_repos ';
Chomp $ youngest;
If ($ youngest EQ $ previous_youngest)
{
Print "no new revisions to backup./N ";
Exit 0;
}
My $ first_rev = $ previous_youngest + 1;
Print "backing up revisions $ youngest.../N ";
My $ svnadmin_cmd = "svnadmin dump -- incremental -- Revision $ first_rev: $ youngest $ svn_repos> $ backup_dir/$ next_backup_file ";
'$ Svnadmin_cmd ';
Open (log, "> $ backup_dir/last_backed_up"); # record the Backup Version Number
Print log $ youngest;
Close log;
# If you want to save space, execute the following compression script.
Print "compressing dump file.../N ";
Print 'gzip-G $ backup_dir/$ next_backup_file ';

4) configure the/etc/crontab file
Configure the/etc/crontab file, specify to execute weekly_backup.pl every Saturday, and specify to execute daily_backup.pl from Monday to Friday;
The specific steps will not be long-winded.

5) Backup Recovery check
Perform the following steps during the restoration check at the end of the month or when a disaster occurs: the recovery sequence is from the lower version to the higher version one by one. That is, first, recover the latest full backup weekly_full_backup.071201 (for example), then restore the Incremental Backup daily_incremental_backup.071202 next to this file, and then restore the backup 071203 of the next day, and so on. As follows:
User1> mkdir newrepos
User1> svnadmin create newrepos
User1> svnadmin load newrepos <weekly_full_backup.071201
User1> svnadmin load newrepos <daily_incremental_backup.071202
User1> svnadmin load newrepos <daily_incremental_backup.071203
....

If Gzip is used for backup, you can combine the decompression and restoration commands during restoration and write them as follows:
User1> zcat weekly_full_backup.071201 | svnadmin load newrepos
User1> zcat daily_incremental_backup.071202 | svnadmin load newrepos
...

(Many references to this part refer to version control.)

2. svnadmin hotcopy full-database COPY method
-------------------------
Svnadmin hotcopy copies the entire database in "hot" mode, including the hook script and configuration file of the database. Running this script at any time generates a secure copy of the version library, whether or not other processes are using the version library.
Therefore, this is a favored backup method.

1) define backup policies

Backup Frequency: perform a full backup every day,
Backup location: the backup directory is named by date. The backup path is/home/backup/SVN/$ {mmdd}
Backup retention period: It can be retained for 10 to 15 days and deleted for more than 15 days.
Backup time: Start at every night
Backup check: After the backup is complete, the check script is automatically run and the report is automatically sent.

2) create a backup script
In your home directory ~ /To create a file, backup. sh:

#! /Bin/bash
Srcpath =/home/SVN/repos/; # define the parent path of the Repository
Distpath =/home/backup/SVN/'date +/% m % D'/; # define the storage path;
If [-d "$ distpath"]
Then
Else
Mkdir $ distpath
Chmod g + S $ distpath
Fi
Echo $ distpath
Svnadmin hotcopy $ srcpath/project1 $ distpath/project1>/home/backup/SVN/cpreport. log 2> & 1;
Svnadmin hotcopy $ srcpath/project2 $ distpath/project2
CP $ srcpath/access $ distpath; # Back Up Access Files
CP $ srcpath/passwd $ distpath; # back up the passwd file
Perl/home/backup/SVN/backup_check.pl # Run the check script
Perl/home/backup/SVN/deletdir. pl # Run the delete script to delete the expired backup.

3) create a check script
Create a Perl script: backup_check.pl at/home/backup/SVN /.
The idea of the backup integrity check is: Run svnlook youngest on the backup database. If the latest version number is correctly printed, it indicates that the backup file is not missing. If an error is reported, the backup is incomplete. I tried to run svnlook youngest if the backup is interrupted.
The Perl script code is as follows:

#! /Usr/bin/perl
# Author: xuejiang
#2007-11-10
# Http://www.scmbbs.com
Use strict;
Use carp;
Use Net: SMTP;

#### Defined the VaR #######

My $ SMTP = net: SMTP-> New ('mail .scmbbs.com ', timeout => 30, DEBUG => 0) | die "cann't connect to mail.scmbbs.com/N ";

My $ bkrepos = "/home/backup/SVN/". & get_day; # define the backup path
My $ ssrepos = "http://www.scmbbs.com/repos"; # define repository URL
My @ repos = ("project1", "project2 ");

My $ Title = "Echo/" is the result of last night's backup comparison with the real database. If the number of backup versions is provided, the backup is successful; if an error message is returned or the number of backup versions does not exist, the backup fails:/">. /Report ";
System $ title | die "Exec failed/N ";
Foreach my $ myrepos (@ repos)
{
My $ bkrepos1 = $ bkrepos. "/". $ myrepos;
My $ ssrepos1 = $ ssrepos. "/". $ myrepos;
My $ svnlookbk1 = "Echo/" $ myrepos last night's backup version is:/">>. /Report; svnlook youngest ". $ bkrepos1. ">>. /Report 2> & 1 ";
My $ svnlookss1 = "Echo/" $ the latest version of myrepos and the last modification time are:/">>>. /Report; SVN log-r 'head '". $ ssrepos1. ">>. /Report 2> & 1 ";
System $ svnlookbk1 | die "Exec failed/N ";
System $ svnlookss1 | die "Exec failed/N ";

}

My $ body = "Echo/" ================================== ========================================================== =/">. /Report ";
My $ Bottom = "Echo/" backup location: From http://www.scmbbs.com ". $ bkrepos."/">./Report ";

System $ body | die "Exec failed/N ";
System $ bottom | die "Exec failed/N ";

###### Report the result ####

Open (SESAME, "./Report") | die "can not open./Report ";
My @ svnnews = <sesame>;
Close (SESAME );
Foreach my $ line1 (@ svnnews)
{
Print $ line1. "/N ";
}

My @ email_addresses = ("SCM/@ list.scmbbs.com", "leader1/@ scmbbs.com", "leader2/@ scmbbs.com ");
My $ to = join (',', @ email_addresses );
$ SMTP-> mail ("SCM/@ scmbbs.com ");
$ SMTP-> recipient (@ email_addresses );
$ SMTP-> data ();
$ SMTP-> datasend ("to: $ to/N ");
$ SMTP-> datasend ("from: svnreport/@ scmbbs.com/N ");
$ SMTP-> datasend ("Subject: SVN backup check report". & get_today. "/N ");
$ SMTP-> datasend ("reply-to: SCM/@ scmbbs.com/N ");
$ SMTP-> datasend ("@ svnnews ");
$ SMTP-> dataend ();
$ SMTP-> quit;

#############

Sub get_today
{
My ($ sec, $ min, $ hour, $ day, $ month, $ year) = localtime (Time ());
$ Year + = 1900;
$ Month ++;
My $ today = sprintf ("% 04d % 02d % 02d", $ year, $ month, $ Day );
Return $ today;
}
Sub get_day
{
My ($ sec, $ min, $ hour, $ day, $ month, $ year) = localtime (Time ());
$ Year + = 1900;
$ Month ++;
My $ today = sprintf ("% 02d % 02d", $ month, $ Day );
Return $ today;
}

4) define the delete script

Because it is a full backup, the backup should not be retained too much. You only need to retained the last 10 days. For backups with more than 15 days of history, they can be deleted.
Create a Perl script under/home/backup/SVN/: deletdir. pl
(Note: deleting a SVN backup database is not as easy as deleting a common file)
Script code please refer to my another post: http://www.scmbbs.com/cn/systp/2007/12/systp6.php

5) modify the/etc/crontab file
In this file, specify to execute the "backup. Sh" script at every night.

3. svnsync backup
-----------------------
See http://www.scmbbs.com/cn/svntp/2007/11/svntp4.php
Using svnsync for backup is simple as follows:
1) Create an empty database on the backup machine: svnadmin create project1
2) Change the hook script pre-revprop-change of the database (because svnsync wants to change the database's attributes, that is, back up the attributes of the source database to this database, so enable this script ):
Cd smp/hooks;
CP pre-revprop-change.tmpl pre-revprop-change;
Chmod 755 pre-revprop-change;
VI pre-revprop-change;
Comment out the three sentences after the script, or simply make it into an empty file.
3) initialization. No data is backed up yet:
Svnsync init file: // home/backup/SVN/svnsync/project1/http://svntest.subversion.com/repos/Project1
Syntax: svnsync init {URL of the library you just created} {URL of the source library}
Note that the local URL contains three slashes :///
4) Start backup (synchronization ):
Svnsync sync file: // home/backup/SVN/svnsync/project1
5) create a synchronization script
After the backup is complete, create a hook script for synchronization. Create/modify the post-commit script under/hooks/in the source library, and add a line in it. The content is as follows:

/Usr/bin/svnsync sync -- non-interactive file: // home/backup/SVN/svnsync/project1

You may have noticed that the above backup seems to be a local backup, not a remote backup. In fact, I mount the remote backup machine (see the mount command) to the svn server. Logically, it looks like local backup, physically it is actually a remote backup.