One-stop solution for integration test Code coverage collection based on cloud technology

Author: Sun Tian, Shi Junjuan, Shenyanyu, Qing Guo

Background

code coverage is an important indicator to measure the effectiveness of product testing. Getting code coverage for unit tests is relatively straightforward. However, Testers of Web applications often worry about collecting code coverage for integration testing or end-to-end testing. The main reason is that testers are often unfamiliar with the technology in this field, and the existing scheme is more complicated and error prone. For example, there is a program that is not very automated and requires users to manually modify many places. We investigated and decided to develop a cloud-based, automated, one-stop solution to collect code coverage for end-to-end testing. The scheme we developed is named "Icoco" (The name means integration test code Coverage (Integration)).

jacoco because icoco jacoco to develop. jacoco A very prominent feature is that it is able to collect code coverage by dynamically injecting instructions in the process of running the code by an agent. Users do not advance any modifications in the project's pom file or other configuration file. All they need to do is just web container ( tomcat "-javaagent:${dir}/jacocoagent.jar=output=tcpserver,port=*,address=*" of course, the directory, The port needs to be specified. Also, the results of code coverage are easy to download because the Jacoco agent initiates a TCP server. See appendix [3] For more information about Jacoco.

the use of continuous integration tools to run test scripts is currently a very common practice. We are currently using the Jenkins server. the CI job is created to perform the test task. Our solutions are seamlessly integrated with the Jenkins server. For more information about Jenkins , see Appendix [1] and [2]

Overview

Not only did we take advantage of the capabilities of Jacoco, but we also leveraged the API of our in-house cloud platform to address application installation and deployment issues.

As mentioned above, since all the testing work is done on the Jenkins server, we decided to develop a Jenkins plugin to do all the code coverage collection work. The Jenkins plugin is easy to integrate with existing test tasks. Existing test tasks do not need to make any changes, just create a new CI job that uses Icoco, and configure the link of the first-use test task. For a plugin on how to develop Jenkins, see appendix [4]. The design requirements for Icoco are as follows:

1) Be able to automatically deploy the tested application to the tested server.

2) be able to download Jacoco agent and modify the Web container startup parameters to load the agent;

3) Ability to merge the results of code coverage for sub-projects.

4) can easily be extended to support different kinds of Web applications.

5) All the above work can be done automatically.

In addition to the above functional requirements, we can reuse existing libraries or services as much as possible in the development process to avoid "reinventing the wheel".

The solution is not only capable of supporting the testing of individual applications, but also supporting the testing of multiple applications.

Is the Icoco configuration interface diagram. We can see that the configuration is very concise.

Figure1

Is the display of code coverage on sonar for integration testing. As you can see, Sonar brings together the code coverage of the integration tests and the code coverage of the unit tests well. At present, the company requires all integration testing and end-to-end testing to collect code coverage, Icoco to help testers do a good job.

Figure2

Icoco Basic Work Flow

1. If necessary, deploy the Web app to the server. The user needs to compile and upload the installation package before deployment. This step is an optional step. This step can be ignored if the user is testing against the current version on the server.

2. Download the Jacoco agent, modify the startup parameters, and restart the Web app to start the Jacoco agent.

3. Trigger the test's CI job for end-to-end testing. During the test, the Jacoco agent records the code coverage.

4. Download the code coverage results using the Jacoco maven plugin.

5. Merge code Coverage results, if necessary.

After the code coverage results are generated, you can use Sonar's Jenkins plugin to upload the structure to the sonar server for test quality monitoring, if necessary.

Architecture and implementation Details

The implementation of Icoco is the Jenkins plugin. Its architecture is a typical three-tier architecture. It is important to note that if Jenkins plugin is required to work on a distributed Jenkins system, there is some special handling. For details, refer to the reference documentation [4]. Because there are some intra-company APIs, the source code for this project will not be exposed. Some code examples are given below as a reference.

1. The presentation Layer Icoco interface allows the user to configure various parameters, such as whether to deploy, test CI job links, etc.

Jenkins implements plug-in interface development through a configuration file called a jelly file. Here is an example code. Refer to the reference documentation [5] for specific details.

<j:jelly xmlns:j= "Jelly:core" xmlns:st= "Jelly:stapler" xmlns:d= "Jelly:define" xmlns:l= "/lib/layout" xmlns:t= "/lib/Hudson" xmlns:f= "/lib/form" >

<f:entry title= "name" field= "Name" >

<f:textbox/>

</f:entry>

</j:jelly>

2. The main entry point for the logic of the control layer Icoco is the Perform method in the Icocobuilder class. The Icocobuilder class inherits the Hudson.Task.Builder class. Various functions, such as deploying, installing JACOCO agents, triggering tests, collecting test coverage, and so on, are all implemented in the Perform method. The values of each parameter of the interface input are passed to the corresponding member variable in the Icocobuilder for use by each function.

3. interface Layer interface or service is called when various functions are executed.

A) The API responsible for deployment in the cloud is called during deployment;

b) The cloud management interface was called to download and install the Jacoco agent without using SSH direct link. The main reason is to avoid entering the user name, password.

c) The API of the Jenkins server is called to trigger the CI job of the test;

D) The API of the Jgit library is used to access the GIT server to download source code for generating code coverage reports. The code examples are as follows:

public void Downloadsourcefromgitrepo (File localrepodir, String giturl, String commitid) throws IOException {        if (loc Alrepodir.exists () = = True) {            fileutils.delete (Localrepodir, fileutils.recursive);        }        Git Localrepo = null;        try {          Localrepo = git.clonerepository ()                      . Seturi (Giturl).                      setdirectory (Localrepodir)                      . Call ();              Localrepo.checkout (). Setstartpoint (Commitid). SetName (Commitid). Call ();        catch (Exception e) {            runtimeexception re = new RuntimeException ("Git clone failed in method Downloadsourcefromgitrep O ");            Re.initcause (e);            throw re;        }         Finally {            if (Localrepo! = null) {                localrepo.close ();}        }                 Outputlogger.remotelogging ("Downloading source code done.");

e) The Jacocomaven plugin is called to generate the final code coverage report.

The following is an example of the invocation code for the Jacoco Maven plugin:

-q-dmaven.test.skip=true-denforcer.skip=true Org.jacoco:jacoco-maven-plugin:0.7.0.201403182114:dump- djacoco.destfile=target/"+ destfile +"-djacoco.port=8084-djacoco.address= "+ Host +" Org.jacoco:jacoco-maven-plugin: 0.7.0.201403182114:report-f "+ Pompath;

f) Merge the code coverage of the subproject with the download results. Merge code coverage calls the Jacoco API. The code examples are as follows:

private static class Mergecccallable implements Callable<boolean, ioexception> {/** * */private static final lo    ng Serialversionuid = 1l;private String rootfolder;    Public mergecccallable (Final String rootfolder) {this.rootfolder = RootFolder + file.separator; }public Boolean Call () throws IOException {Mergejacocoresult (); return boolean.false;}  public void Mergejacocoresult () throws IOException {list<file> flist = Getccfilelist (); if (flist.size () = = 0) {throw New RuntimeException ("No Code coverage result file has been found!");} Final Execfileloader loader = new Execfileloader (); for (final File f:flist) {loader.load (f);} Final file Disfile = new file (RootFolder + "target/icoco.exec"); Loader.save (Disfile, false);}  Public list<file> Getccfilelist () {final list<file> res = new arraylist<file> ();//Collect result files In the target folder of root folderstring rootfld = RootFolder + "target"; outputlogger.remotelogging (ROOTFLD); File Folder = new file (ROOTFLD);  file[] list = Folder.listfiles (), if (list! = null) {for (final File f:list) {if (F.getname (). Matches ("^icocotmp.*.exec")) {Res.add (f);}}}        return res;} }

Figure4

Summary

So far, this solution has been applied to end-to-end testing and has achieved good results. Download, installation configuration is very simple, users do not have a deep understanding of the collection details of code coverage, saving a lot of time.

Resources

1. The official web site forjenkins:http://jenkins-ci.org/

2. The description about Jenkinson Wikipedia:http://en.wikipedia.org/wiki/jenkins_%28software%29

3. http://www.eclemma.org/jacoco/

4. Http://ccoetech.ebay.com/tutorial-dev-jenkins-plugin-distributed-jenkins

5. Https://wiki.jenkins-ci.org/display/JENKINS/Basic+guide+to+Jelly+usage+in+Jenkins

One-stop solution for integration test Code coverage collection based on cloud technology