Maven Combat (iv)--A continuous integration practice based on MAVEN

Martin's "Continuous Integration"

I believe a lot of readers are like me. The concept of first exposure to continuous integration was the famous article "continuous integration" from Martin. The article was first published in September 2000, after which a revision was made in 2006. It clearly explains the concept of continuous integration. and summed up 10 practices, which were:

• Just maintain a source code repository

• Self-initiated construction

• Let's build a self-test

• Each person submits code to the backbone every day

• Each commit should build the backbone on the continuous integration machine

• Maintain a high-speed build

• Test in a simulated production environment

• Make it easy for everyone to get the latest running files

• Everyone can see the progress.

• Self-initiated deployment

The original article has been more than 10 years, this is a very long time in the software industry, but we can see that Martin summed up these practices are still shining, there are still a lot of teams are trying to practice them and get rich returns, There are, of course, many teams who refuse to practice continuous integration for various reasons and cannot appreciate the benefits.

From these 10 practices we can find a lot of popular open source tools shadow. such as version number Control tool CVS, SVN, git, self-motivated build tools maven, Ant. Self-motivated testing framework JUnit, TestNG, and continuous integration of Servercruisecontrol and Hudson, and so on.

In fact, whether or not you practice continuous integration, using a very large number of tools alone can be of great value. One of the big implications of continuous integration is that it introduces an effective process. To allow these tools to blend organically. and promote each other.

About continuing to integrate another book that won the Jolt Award. "Continuous integration-the way to software quality improvement and risk reduction."

But whether it's Martin's article or the book. has not elaborated on the details of continuous integration using MAVEN as its own proactive build tool. The purpose of this article is to introduce some of the practices of continuous integration based on MAVEN, and I hope that these experiences can be helpful to the reader from a detailed point.

Set up a private maven warehouse

Martin's article does not cover the content of dependency management. But in the world of Java. Dependency management is a problem that developers have to face. Whether the external open source class library relies on, or the inter-module dependencies within the project, need to be effectively managed. The ability to say that dependency management is one of the core elements of continuous integration.

Maven effectively overcomes this problem through its dependency management mechanism and a central repository that is available anywhere, and users simply need to declare the dependencies required for the project in the Pom. Maven can proactively parse dependencies from the warehouse at the time of construction.

Just this is not enough. We know that the greatest benefit of continuous integration is to reduce risk, simply by exposing the problem early enough to allow developers to find and fix it early, thus reducing the cost of remediation. However, it is time-consuming to assume that everyone is repeatedly downloading dependencies from a central repository. The integrated feedback period will certainly be extended.

I've heard a lot of people complaining "maven is downloading the entire internet!

”。

Build Fast!

Continuous integration feedback is fast. Maven You can't slow down this process.

Fortunately, the open source world has a very good solution, and the problem can be solved simply by using MAVEN warehouse manager software like Nexus to build a private maven repository. The principle is very easy, the MAVEN repository located in the local area network can proxy all the external warehouses, thus preventing all people from downloading dependent files from the Internet repeatedly.

In this way MAVEN parsing relies on the local area network only, the construction speed is greatly accelerated.

For example, when the junit-4.8.2.jar first person requests a private warehouse, the private repository is downloaded and cached from the central repository if it takes 10s. When other people need it junit-4.8.2.jar , the private warehouse uses the cached file directly, which can be a time-consuming 1s. If there are 100 developers using the file, the time saved is ten-(+ 1) = 891s. The actual amount of reliance may be hundreds of thousands of times, and the time saved will become considerable.

One might say that I am also fully able to add Project dependencies to version number control. This is even mentioned in the "Effective Procedure Ape", in the 5th chapter of the book, "Dry version number Control", Neal Ford has the phrase: "All the things used to build the project should be put into the version number control, including binary files (class library. Framework, jar files, build scripts, etc.) ". The author further explains its purpose by doing so to ensure that the project is not affected by external factors such as dependent version number changes. Even lost), to ensure the stability of the construction, the author also mentioned at the same time the General Version number Control tool to handle the performance of binary files. Put aside this concluding practice and carefully consider its purpose, we can find out. The same as private Maven repositories can guarantee the stability of the build. It also avoids potential performance issues caused by the version number Control tool to handle binaries. So. I venture to say that the practice that Neal Ford has put out is out!

The purpose of the private MAVEN warehouse is not limited to this, it can greatly promote the efficiency of project integration by combining its own proactive deployment and Maven's snapshot mechanism.

In a modular development environment. Everyone has their own responsibilities, focusing on the modules they are responsible for, and the rules for continuous integration are that before you control the commit code to the version number. It is necessary to ensure that the local build is not problematic. The general practice is to update the code of all modules and build them. But do you really need to build modules that you don't actually care about? And don't talk about an error once you build someone else's code. You tend to be overwhelmed by this practice by adding time to local builds at the same time.

Maven has the snapshot version number concept, which is designed to allow you to build a temporary version number. For use by others on the team. So they don't have to care about their dependencies at the level of the code. The private Maven repository acts as an intermediary. and continuous Integration Server has one more responsibility. Each time it builds a module successfully, it should publish the module's snapshot version number to the MAVEN repository. Now. You don't have to build someone else's code. MAVEN can take the initiative to help you resolve download dependencies from the private warehouse to the latest snapshot (force update using the- u parameter of the mvn command). Note that, in addition to continuous integration Server, no one else should advertise the snapshot version number to the Maven repository, because only the environment with continuous integration Server can be trusted, you can successfully run MVN clean install locally Does not mean that the command succeeds on the continuous integration server because each person's local environment is different, so the success of the integration should be based on continuous integration of server, but only after the integration is successful. Snapshot is capable of being deployed to a private warehouse for others to use.

In view of the above reasons analysis. I think in the context of continuous integration based on MAVEN. The importance of private Maven repositories is never too important to be overemphasized.

The Right Integration command

The question of how to integrate a project using the MVN command on the continuous Integration server, at first glance, is the obvious answer, not mvn clean install ? In fact, better integration commands are slightly more complicated. Here are some summaries:

• Don't forget to clean: Clean ensures that the last build output does not affect this build.

• using deploy instead of install: the built-in snapshot output should be proactively deployed to a private Maven repository for others to use, as discussed in detail earlier.

• Use-u parameter: This can force maven to check all snapshot dependent updates, ensuring that the integration is based on the latest state. Assume that there is no such parameter. Maven checks for updates by default in days, and the frequency of continuous integration should be much higher.

• using the-e parameter: Assuming that the build has an exception, this parameter allows MAVEN to print the complete stack trace for easy analysis of the cause of the error.

• using-dmaven.repo.local: Assuming that continuous integration server has a lot of tasks, each task uses a local repository, and the download relies on the local repository. To avoid this multi-threaded use of local repositories may cause conflicts. The ability to use -dmaven.repo.local=/home/juven/ci/foo-repo/ to assign a local warehouse to each task.

• using the-B parameter: This indicates that MAVEN uses batch mode to build the project, avoiding some pending states that need to be manually participated and interacted with.

In summary, the integration command on the continuous integration server should be mvn clean deploy-b-e-u-dmaven.repo.local=xxx . In addition, it is a good practice to regularly clean up the local MAVEN repository for continuous integration server. This avoids wasting disk resources, nearly all of the continuous Integration Server software supports local scripting tasks, you can write a single line of shell or bat scripts, and then configure your own initiative to clean up the warehouse in days. It is important to note that the premise is that you have a private maven repository. Otherwise, it's a nightmare to download all the dependencies from the Internet every time.

Use a good profile

Assuming you don't have to consider a variety of environments, and your own proactive test (including integrated testing) runs fast, you don't have to build multiple integration tasks for your project. But the real situation is. Integration may take into account a variety of environments, such as development environment, test environment, product environment. And when the project gets bigger. As more tests are being made, it becomes increasingly unrealistic to control the build time within an acceptable range (say, 10 minutes).

"Continuous integration-the way to software quality improvement and risk reduction" describes a solution called staged build (staged build) , such as you can divide the build into two parts, the first part contains the tasks of compiling and unit testing that can be completed at high speed, The second part contains time-consuming tasks such as integrated testing, and the second part of the integration is triggered only after the first part has been successfully completed. The point of doing this is to keep the feedback on the continuous integration as fast as possible.

MAVEN's profile mechanism can be very well supported in a phased build. For example, with Maven Surefire Plugin, you can name the Unified Unit Test **UT . The unified integration test is named **IT . Then configure MAVEN Surefire plugin to simply perform the unit test and then write a profile named Integrationtest. Configure MAVEN Surefire plugin to perform an integrated test.

It then builds the project in stages, with the first build as MVN clean install-b-e-u , and the second build task as mvn clean deploy-b -e-u-pintegrationtest 。 The previous build succeeds and then triggers a second build before it is deployed to the MAVEN repository. It is worth mentioning that Maven Surefire plugin can support JUnit 3, JUnit 4 and testng very well, and you can divide the unit test and the integrated test in the way that suits you best.

There is also a common phased build case where a Maven site is generated, and using the mvn clean site to build the site is often time-consuming and resource-intensive, and the ongoing review phase of the task corresponding to continuous integration, often without the need for very high integration frequencies. You will want to check the source code changes and compile tests every 10 minutes, but very few people would like to have the system generate a test coverage report, Checkstyle report, etc. every 10 minutes, so it is reasonable to use a lower frequency. For example, every day, this can avoid unnecessary resource consumption. More importantly, this will not slow down the feedback content that would have been very fast for compilation and unit testing.

In other cases where the system needs to be integrated based on different environments, Maven's attribute mechanism, resource filtering, and the previously mentioned profile are needed. The reason for the length is no longer here.

Summary

Continuous integration is one of the most important agile practices, but there are few articles detailing how to practice continuous integration in a MAVEN-based environment. This article describes some best practices for this topic. It includes setting up a private warehouse, using the correct integration commands, using technology such as profile to handle staged construction, and so on. This article aims to make the majority of Maven users aware of the existence and importance of these practices. Details such as Nexus installation configuration, Maven Surefire plugin configuration, or profile configuration usage are not explained. Suppose you would like to see a more detailed introduction to my "Maven combat" book.

In addition to the above, the book also details how to use the Hudson (perhaps renamed Jenkins), the most popular open source continuous integration Server.

Of course. Let's say you have experience with MAVEN and continuous integration, and don't hesitate to share it.

Original address: Http://www.infoq.com/cn/articles/xxb-maven-4-ci

Maven Combat (iv)--A continuous integration practice based on MAVEN