Continuous integration tool selection

Continuous Integration
Integration) as the cornerstone of Agile programming is now widely used by the vast majority of development teams. The continuous integration tool is now blooming in different ways. This article mainly compares
Several common CI servers in the Java field (because the company needs to unify the company's Ci
Server ). For more tools, see here: http://confluence.public.thoughtworks.org
/Display/CC/Ci + feature + matrix, this webpage focuses on most popular Ci
Server, but I found that a lot of content lags behind the features of the actual product, so if you want to compare it, you may have to go to the product site to see it, it is better to download and run it.

 

In this article, I mainly compare the following types of Ci servers, which are selected by various project teams in the company at present (there are a lot of versions, and some open source is used for multiple domestic projects, in contrast, foreigners often use commercial versions. I added cruisecontrol and teamcity because they are very famous .) :

 

• Cruisecontrol (http://cruisecontrol.sourceforge.net /)
• Hudson (https://hudson.dev.java.net /)
• Luntbuild (http://luntbuild.javaforge.com /)
• Teamcity, http://www.jetbrains.com/teamcity)
• Anthill Pro (http://www.anthillpro.com /)
• Bamboo (http://www.atlassian.com/software/bamboo)
• Quickbuild (http://www.pmease.com /)

In the field of continuous integration, the cruisecontrol and luntbuild of opensource are old-fashioned, especially the cruisecontrol.
Thoughtworks, this is Martin
Fowler's nest. As a rising star in opensource's continuous integration, Hudson has now surpassed these two predecessors. Currently, I am afraid it is the most widely used CI.
Server. The last four are commercial CI servers, where teamcity comes from jetbrains and jetbrains is a well-known ide
Intellij company. Bamboo is the famous bug tracking tool Jira and Wiki.
Confluence company Atlassian. Anthill also belongs to continuous
Quickbuild is the commercial version of luntbuild. I will focus on quickbuild, because
Luntbuild seems to be slow, and quickbuild now has free community.
Edition, full-featured, only limited by the number of configurations. In these commercial versions, teamcity has the highest market share. Companies tend to use commercial services.
So I will focus on the last four types of containers, and compare them with cruisecontrol and Hudson. Both teamcity and quickbuild have their own free versions,
If you are interested, you can check it out.

Function Comparison

Ci
Server is essentially a timer scheduler. We configure a series of projects, set a timer, let it do some work, and then notify everyone. So many companies use the so-called home-
The made tool uses cron + ANT/Maven for continuous integration, which can achieve the simplest CI functions. In addition to the basic compilation and notification functions
In addition, we have many other requirements. In our company, we mainly consider the following points when selecting CI Server:

 

• Facilitate unified management of the company (about 200 + projects need to be managed in a unified manner)
• Manage the project process: daily build-> QA build-> release build
• Access to company Active Directory to manage User Permissions
• The support of Continuous testing means that the test of the project should generate detailed reports and collect the statistical data of the test as the analysis and consideration of the project.
• With support for continuous code quality analysis, you can process coverage reports generated by projects, static analysis reports of code, and collect statistics of these reports for Project analysis and consideration.
• Integration with SCM tools, our company mainly has three VCs, clearcase, subversion and starteam
• Integration with other tools, such as bug tracking tools and IDE integration.

First, let's take a look at how to install Ci

Installation is the first step in the beginning.
Server has a preliminary impression. According to their respective manuals, it will soon be installed. I basically chose the standalone version, that is, not configuring the database, using the built-in version, or
Deploy to tomcat or other containers, basically every CI
Server is very simple. So we didn't see any good or bad. Here, I have to mention anthill. It's a bit childish. to download, I have to submit a request, and then
Download, install, and get bored.

Configuration item

In most CI servers, most of them are projects or projects.
Group is used for management. Only luntbuild and quickbuild are different. They use the configuration term, meaning a configuration. In configuration 1
For a typical project, you can only process the following basic processes: checkout, build, and publish.
Artifacts, these tools are complete very well and very simple. After I use them, I think teamcity's navigation is the most convenient and clear at a glance. Luntbuild and
Quick build is slightly less user-friendly in this regard, and neither of these tools uses the wizard mode.

 

Next, I will configure 50 test items in the experiment, which will test a CI
Server management capabilities (because we have many projects ). After using it, I found that quickbuild is the most practical for me. Because it uses configuration instead
Project, and quickbuild is the only CI server that supports tree structure configuration. I can configure configuration to Team
A, Team B
..., And then configure any number of child nodes and Sun nodes for each team according to the actual situation (note that the number of configuration nodes is in the quickbuild
Community
In the edition, it seems to be a maximum of 16. In addition, the inheritance relationship of quickbuild is also very convenient to use. If you want to manage a large Ci
Server, without such inheritance is simply a torment for me. For example, if I used Hudson to configure 50 projects, I spent a lot of time, but I used quickbuild, and I only used it.
In an hour, there are only three configuration items in my actual configuration (including the actual step definition), all others are inherited, and then the parameters are modified. If we need to batch
When modifying a series of mappings, we usually need to modify the settings of the parent node. Teamcity support
The concept of project group is similar to a tree, but it is not complete yet. It can only be divided into two levels of relationships, that is, project
Group and project. In addition, the inherited functions of quickbuild are not seen in other CI, and some are just copies similar to teamcity.
Project. Quickbuild is far ahead of other Ci in terms of replication capability.
Server, which can copy the entire subtree, which means that I can configure a template used by the company.
Configuration tree, and then copy department a, department B, Department C, and so on. For the differences between different projects, you can use variables to control them! Teamcity Configuration
The convenience is really nothing to say, It is very intuitive, the coolest is tests like JUnit, nunit, and even ant scripts do not need to be written, it can directly find the unit in the project
Tests, which has never been seen in other tools. As for cruisecontrol, Hudson, and bamboo, they are well-planned and have no bright spots. This ring
Section, quickbuild and teamcity won.

 

In addition, to configure a project, you need to configure the process management for continuous integration of the project. Here, the process is basically as follows: daily build-> QA
Build-> integration build-> release build. Daily
Build, as its name implies, is once a day. It is managed by the development team to ensure smooth project execution. After a while, development
The team must submit the application to the QA Team for testing. Generally, it is about two weeks to one month, which varies with the project size. After the QA test is completed, if there are no major problems, submit the application
Integration test to ensure normal operation in the simulated real environment. Finally, if there is no problem, perform the release
Build to form the release version. For some teams in the company who use Agile programming, the so-called commit test needs to be added.
Build, that is, the developer automatically triggers a build during every checkin operation to ensure that the build will not be destroyed by this checkin (including
Damage Unit Tests and code quality ). This is also called the continuous testing and continuous code.
Quality Analysis, all of which use JUnit, nunit, checkstyle,
It is implemented by tools such as PMD, cobertura, and fxcop. As we will talk about later, we will skip it here. In this step, I prefer
Pro and quickbuild, both of which emphasize processes. In particular, anthill pro uses them as selling points. Anthill
Pro defines this process in the workflow mode. A project can define multiple workflows, corresponding to our case, which is to define daily.
Build workflow, define QA
Build workflow, and so on, and then select different workflow for the purpose during promote. While quickbuild uses existing
Configuration, define different configurations, and then define one or more
Promote configurations. To use promote, click the promote button of a build to send its promote to the specified
Configuration. The concept of using the anthill model is very clear, because we want to perform process management, so workflow will sound more comfortable.
Easy to accept. While quickbuild binds it to configuration, which is relatively simple to use, but it takes some trouble to find it, at least for me.
Hudson also has similar process management, but it is automatic, and promote here requires people to review, that is, people to participate, determine which version to use
Promote, so here we are not very suitable.

 

In the configuration project process, I personally feel that quickbuild is flexible, which can be simple configuration or very complicated configuration, and it is very convenient to configure. The term is a little different from that of other Ci servers.

 

Build Function

The most important part of Ci server is the build function, including SCM connection, user permission management, and build tool support. First, let's look at the support of SCM.

SCM support

In these CI servers
Pro and Hudson support the most types, especially Hudson. SCM is supported on the public. For common
Subversion, CVS, clearcase, starteam, and sourcesafe are all supported. The performance in the previous project is better.
Quickbuild is one of the least supported in SCM. It does not support git, Team Foundation
Server, which is already popular with SCM, is a pity. However, when quickbuild supports SCM, many variables are supported.
Ci
Server is the most flexible one. It can use variables to configure SCM URLs. Others define a basic URL and then define their respective SCM for different projects.
Repository. Quickbuild also has its own quickbuild.
Repository is used to transmit artifacts in different configurations. It is also very convenient to use. For example, we need to use
Artifacts, you can define this repository. Of course, this function can also be achieved through Maven repository.
Teamcity also provides a similar mechanism, but teamcity's repository is actually an Ivy extension.

 

SCM data is reflected in these CI servers, from the change of each build
Sets to historical statistics. This shows that everyone attaches great importance to the collection and analysis of such data. Specifically, teamcity can directly call IDE from the web page to open these changed files:
Big highlight: after all, it's an intellij company!

User Management

This is basically a required function for each CI server. Basically, it can be used for both built-in Database Management (Hudson seems useless) and LDAP server connection. I just tested it for a moment. Without further understanding, I have no right to speak.

Dependencies management of build (dependent builds)

In actual projects, we often see dependencies between projects. For example, project a depends on Project B, and Project B depends on Project C. So when we want to compile project a, we need to compile
Translate C project, then compile B project, and finally compile a project. The advantage of this is that we always use the latest code to compile a version, if something goes wrong, I
You can easily know which project break is used for the entire build process. This function basically includes all of these Ci
Servers are provided, and their capabilities are different. Teamcity is the weakest one here. It can only achieve dependency management of artifacts in different projects by defining Ivy,
However
Both pro, bamboo, and quickbuild provide two types of dependency management: artifacts and project dependency management. However
Teamcity has another killer, that is, the ability to import projects. It supports direct import from intellij projects and Maven projects to create such dependencies.

 

Distributed build pool

Because the company has a wide range of projects and a wide range of platforms, a project needs to be distributed to different platforms for compilation and testing. At this time, a build is required.
Pool, basically all the above CI servers have supported this distributed build pool, the essence is to use the grid
Computing Technology for management. That is, a build server carries a group of Build
Then, the build tasks are distributed to different agents for execution. I have to like another quickbuild here (haha, this quickbuild seems like
Surprise) In fact, the agent of quickbuild is no different from that of other companies, but it is just a computing
Unit, the key is to configure a configuration in quickbuild, which uses the concept of STEP (this quickbuild has many terms ),
This step is in
Pro also exists, the key is that this step can be distributed, that is, when I configure a project, you can define a series of parallel distributed steps, so that for management and collection
Artifacts is very convenient. We can define test on Windows, test on Mac, and test on.
Linux, and then set the type of agent required to run these steps. quickbuild can distribute these tasks to the agents of these platforms to run.
. Other companies may be charged by using teamcity. A build can only run on one agent. To achieve the same effect, I need to define three projects,
Then let the three projects run on different agents, and finally define another project so that the project can collect their artifacts, which is very troublesome. Bamboo and
Anthill is similar to teamcity. However, Hudson has a weak capability in this field. I personally feel that it is not as powerful as other products, and it is more complicated to use.

 

Report functions and statistics

The above Ci
The server provides the report function and statistical function. In this field, Hudson has no suspense that it supports the most types of reports and the most comprehensive (who is called our opensource?
Some are developed by people ). Bamboo supports the least types of reports, but there are also many third-party plugins to choose from. Several of the reports we care about are supported by various companies.
Quickbuild reports in the show me the most gorgeous, but it seems to be referring to Google
From the perspective of analytics, analytics is simply a flip. Quick build and teamcity are the most convenient to use.
Links can be filtered. In terms of statistics, the tests statistics are complete, which also reflects the test
Driven is deeply rooted in the hearts of the people. In the test
In driven aspect, teamcity is the top driver. Thanks to the experience in intellij development, teamcity can not only automatically find the unit in the project
Tests (you do not need to call JUnit task in ant script, or call surefire in Maven), and test
Cases, which can be automatically run in the next build, so that a build can be prevented from running for a long time before it finds that the test that failed last time has not been corrected. Strong!

 

In addition, I like the dashboard of the build in quickbuild very much. The current status of a project is clear at a glance.
Tests has succeeded, failed, fixed, and not fixed. In short, the information is rich, but the configuration is a bit complicated. I need to report a report to add it.
Step. If teamcity can be achieved, it will be perfect.

 

For other Ci servers, there are not many highlights (in fact, they are also very strong, but in comparison, I think teamcity and quickbuild are stronger and better ).

 

Integration with third-party tools

In integration with third-party tools, Hudson is far ahead and the most plug-in all CI servers. Can be used with Facebook or Google
Calendar and Twitter, basically all you can think. However, many plug-ins do not have much value for us. Bamboo is working with several of its own products
Product integration, such as Jira, wiki, and clover. These are all useful in our company and are ideal for this.

Price

I have to consider the price. It seems that someone has said that price is nothing, but price is
Everything, especially in the age of financial crisis. There is no doubt that opensource will always be the best. In commercial use, quick build is the most convenient
Yes, it uses site
License, a site charge $2999, the most expensive for Anthony. I asked, according to my configuration, it would take $10000 to get started with teamcity.
Yes, $1999 has three agents, but for our situation, it will cost $8000 (it is charged by Agent). bamboo is also very expensive, based on its functions
In my opinion, the cost effectiveness is not very good.

Summary

Taking into consideration various factors, we finally chose quickbuild. Although this product is not very famous, there are many customers who think about it, such as Cisco and HP.
Companies should be trustworthy. In addition, we feel that it still has many highlights, which is extremely convenient for our unified management. In addition, the price is also very good. Of course
If your team is not large, select the Community edition of quickbuild and the professional of teamcity.
Edition is very worthwhile, both of which are free, and the community of quickbuild
The edition function does not have any cropping, but only limits the number of configuration items. It is very suitable for teams with high requirements and not many projects.

 

Well, there are too many things to be discussed. The Ci field is still in a fast development stage. This article is purely a discussion. You are welcome to make a picture. Due to limited time, I don't know much about each product, and errors are inevitable. If I have any accuracy, please let me know.