Measuring test coverage with Cobertura

Cobertura is an open source tool that measures test coverage by detecting basic code and observing what code is executed and what code is not executed when the test package is run. In addition to finding the code that is not tested and discovering bugs, Cobertura can also optimize the code by marking unwanted, executed code, and provide internal information about the actual operation of the API. Elliotte Rusty Harold will share with you the best practices for using code coverage to use Cobertura.

Although Test-first programming (Test-first programming) and unit testing are not new concepts, test-driven development is still the most important programming innovation of the past 10 years. Some of the best programmers have been using these technologies for the past Half-century, but only in recent years have these technologies been widely seen as the key to developing robust, bug-free software in time and cost budgets. However, test-driven development cannot exceed the level of testing that can be achieved. The test improved the quality of the code, but it was only for the part of the code that was actually tested. You need a tool that tells you which parts of your program are not being tested so that you can write test code for those parts and find more bugs.

Mark Doliner's Cobertura (Cobertura is covered in Spanish) is a free GPL tool to complete this task. Cobertura measures bytecode in such a way that the tests are monitored by using additional statements to record which rows were tested and which rows were not tested when the test package was executed. It then generates an HTML or XML-formatted report that indicates which packages, which classes, which methods, and which rows were not tested in the code. You can write more test code for these specific areas to discover all the hidden bugs.

Read Cobertura output

We first look at the generated Cobertura output. Figure 1 shows the report that the Jaxen test pack runs Cobertura generated (see Resources). From this report, you can see coverage results from very good (almost 100% in org.jaxen.expr.iter packages) to very poor (completely without coverage in org.jaxen.dom.html).

Figure 1. Jaxen Package-level coverage statistics

Cobertura calculates coverage by the number of rows being tested and the number of branches tested. The difference between the two test methods is not important at the time of the first Test. Cobertura also calculates the average McCabe complexity for the class (see Resources).

You can drill down into HTML reports to understand the coverage of specific packages or classes. Figure 2 shows the coverage statistics for the org.jaxen.function package. In this package, coverage ranges from 100% to idfunction classes in the Sumfunction class to only 5%.

Figure 2. Code coverage in the Org.jaxen.function package

Drill down further into a separate class to see exactly which line of code is not tested. Figure 3 shows the partial coverage in the Namefunction class. The leftmost column shows the line number. The latter column shows the number of times this line was executed when the test was performed. As you can see, line 112th was executed 100 times, and line 114th was executed 28 times. The rows highlighted in red are not tested at all. This report shows that although the method is generally tested, there are actually many branches that are not tested.

Figure 3. Code coverage in the Namefunction class