Daily automatic backup Jenkins_home to SVN

Daily automatic backup Jenkins_home to version control system SVN

Jenkins is a scalable, open source, continuous integration tool and the most popular continuous integration tool of the moment.

Jenkins main functions

• Ease of installation (easy installation)
• Easily configurable (easy configuration)
• Changeset support (change set supports)
• Permalink (Permanent links)
• Rss/email/im Integration (RSS/E-MAIL/IM integration)
• Afterwards tagged (after-the-fact tagging)
• JUNIT/TESTNG Testing Report (junit/testng test reporting)
• Distributed build (distributed builds)
• Document fingerprint printing (file fingerprinting)
• Plug-in support (Plugin)

Plug-in support makes Jenkins customizable and extensible.

Jenkins_home directory Structure

The related files of Jenkins are stored in the file system, and Jenkins_home is mainly used to store these files, such as system configuration files, plugins, configuration files for each job, etc.

The Jenkins_home directory structure is as follows:

+ Config     (Jenkins root configuration) +-*.xml          (other site-wide configuration files) +-Usercontent    ( Files in this directory would be served under your http://server/userContent/) +   -fingerprints (stores fingerprint reco  RDS) +        -plugins (Stores plugins) +-Jobs     +-[JOBNAME]      (sub directory for each job)         + config     (job Configuration file)         +         -Latest (symbolic link to the last successful build)         +-builds +-             [Build_id]
    (for each build)                 +      -build.xml (build result summary)                 +-Log            (log file)                 +-Changelog.xml  (change log) +-workspace      (Working directory for the version control system)

Whether it's the upgrade, migration, or backup or restore of Jenkins, the jenkins_home operation is required, and the backup Jenkins simply backs up the jenkins_home.

In Jenkins_home, some directories are not backed up, such as: Workspace.

Because workspace is the working directory of the version control system, the relevant files in this directory are from the version control system and can be obtained from the version control system.

Jenkins Backup Plugin

Jenkins is extensible and has thousands of plugins to choose from, with two plugins to choose from for backup: Thinbackup plugin and backup plugin.

Thinbackup plugin can automatically back up the global and job-specific profiles (excluding archive and workspace).

Backup plugin can back up jenkins_home and choose whether to back up workspace, builds history, maven atifacts archives, fingerprints, and so on.

Thinbackup plugin and backup plugin are different:

1. Bakup plugin can only trigger backups manually, Thinbackup plugin regular automatic backups.
2. Bakup plugin can back up jenkins_home, you can choose which content needs to be backed up (such as workspace, builds history, etc.), thinbackup The plugin only backs up the most important information (global and job-specific configuration files).

The above two backup plug-ins generally meet the general requirements.

Typically, the continuous Integration Server resource is dedicated, and Jenkins is installed on a dedicated server (such as a virtual machine).
However, if the virtual machine hangs, it is difficult to recover within a short period of time, it is not natural to recover from the backup (all two plugins are native backup).

So in order to deal with this situation, we should back up jenkins_home to other places.

Backing up the Jenkins_home to the version control system

The author chooses to backup jenkins_home to the version control system:

1. The version control system chooses SVN.
2. Automatic backup Jenkins_home to SVN per night
3. In addition, by using SVN and comparing the differences between the two revision, you can view the changes to facilitate traceability

The initialization steps before the backup are as follows:

1. Create a new directory on the SVN repository, such as Jenkins_home_bak
2. Check out the Jenkins_home_bak (checkout) to a location on the JENKINS server, such as E:/jenkins_home_bak

The specific backup steps are as follows:

1. Copy Jenkins_home to Jenkins_home_bak
2. Incorporating a file or directory that is not versioned in Jenkins_home_bak into version control (SVN add operation)
3. Traverse Jenkins_home_bak, if the file or directory is not in Jenkins_home, perform an SVN delete operation on it
4. Last Commit to SVN repository (svn ci operation)

The above steps, if the 2nd step is manual, theoretically is difficult to implement, so if it is a manual backup, you can not perform the 5th step
And these process steps, if manual operation, unusually cumbersome, repetitive, lazy people usually lazy to do repetitive things, so should be automated.

About automating backup scripts, the main step is to implement the 4 steps above:

1. For the 1th step, use the ant script to implement

   • Ant scripts are not just built, they can also be used to build outside
   • Ant's syntax is flexible and has a lot of tasks to choose from.
   • The copy operation uses the Ant copy task, which allows you to use Fileset's excludes to exclude files or directories (such as workspace) that do not need to be copied, which is convenient
   • Ant Copy task can increment copy

2. For steps 2nd, 3rd, and 4th, use PYTHON+SVN client comands to implement

   • Python was chosen because of the recent learning of Python
   • Choosing SVN client Commands instead of PYSVN is because the PYSVN is not installed on Windows, so instead of using SVN client commands

In addition, automated backups can be created on Jenkins to create a job dedicated to scheduled backups Jenkins_home

Specific implementation of the script see: Donhui/jenkins_home_bakcup

The tools and environment used in the above are added:

* Windows 2003 Server* Jenkins 1.592* Python 2.7.8* SVN Server 1.7* svn client commands 1.7（Windows下安装TortoiseSVN时需要手动选择安装）* Ant 1.8.1

Reference

• Https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
• Https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins
• Https://wiki.jenkins-ci.org/display/JENKINS/thinBackup
• Https://wiki.jenkins-ci.org/display/JENKINS/Backup+Plugin
• http://blog.csdn.net/spare_h/article/details/6677435

Daily automatic backup Jenkins_home to SVN