Use Emma to measure test coverage

Emma is an open-source tool used to detect and report Java code coverage. It is not only suitable for small projects, but also for large enterprise projects.

Emma has many advantages. First, you can get it for free and use it for your own project development. It supports many levels of coverage metrics: packages, classes, methods, statement blocks (basic blocks) and rows. In particular, it can determine whether a row is partially overwritten, for example, the condition statement is short. It can generate reports in the form of text, XML, HTML, and so on to meet different needs. Its HTML report provides the drill-down function, and we can link to a method we are concerned about step by step from the package. Emma can be integrated with makefile and ant to facilitate application to large projects. In particular, Emma is highly efficient, which is important for large projects.

Emma traces and records the Running code information by inserting bytecode into the. Class file. Emma supports two modes: on the fly and offline.

On the fly mode, adding bytecode to the loaded class is equivalent to replacing the original application Class Loader with the application Class Loader implemented by Emma.

The offline mode adds bytecode before the class is loaded.

On the fly mode is more convenient and has obvious disadvantages. For example, it cannot generate a coverage report for Classes loaded by the boot class loader, you cannot generate a coverage report for a class that has a unique class loader like a J2EE container. In this case, we can turn to the offline mode.

Emma supports command line and ant.

The command line is generally used in combination with the on the fly mode and can quickly generate coverage reports for simple projects. Emma runs using ant tasks, which is especially suitable for large projects.

The examples provided later in this article mainly demonstrate how to integrate Emma and ant to generate coverage reports in offline mode.

Back to Top

Sample project

Sample project sampleproject is a small project with a numberparser class. The main function is to parse a string into a float type. The following is the directory structure of the entire project.

Figure 1. directory structure of the sample project

Next, we will start to compile the ant script for our project.

Listing 1 sets some attributes, including the paths of source files, binary files, JUnit reports, and coverage reports.

<! -Set the path for storing the Java class after the byte code is injected --> <Property name = "bin. instrument. dir" location = ".../export bin"/> <! -Set the coverage metadata and report path --> <Property name = "coverage. dir" location = "../coverage"/> <! -- Set the JUnit report path --> <Property name = "junitreport. dir" location = ".../junitreport"/> <! -Set the bin path of the topic code --> <Property name = "bin. Main. dir" location = ".../srcbin"/> <! -Set the test code bin path --> <Property name = "bin. Test. dir" location = "../testbin"/> <! -- Set the Source Path of the topic code --> <Property name = "src. Main. dir" location = ".../../sampleproject/src"/> <! -- Set the test code source path --> <Property name = "src. Test. dir" location = ".../../sampleprojecttest/test" /> <! -Indicates the path of the Java class to be injected with bytecode --> <Path id = "classpath. Main"> <Pathelement location = "$ {bin. Main. dir}"/> </Path> <! -Indicates the path of Emma. jar and emma_ant.jar --> <Path id = "Emma. lib"> <Pathelement location = "$ {libs}/Emma. Jar"/> <Pathelement location = "$ {libs}/emma_ant.jar"/> </Path> <! -Enable Emma --> <Property name = "Emma. enabled" value = "true"/>

The $ {bin. instrument. dir} directory stores the class injected with bytecode, and "Emma. lib" points to the location of the Emma resource.

Listing 2 defines Emma tasks for ant

<! -Add an Emma task for ant --> <Taskdef resource = "emma_ant.properties" classpathref = "Emma. lib"/>

Listing 3 compile source code and test code

<target name="compile-src.main"> <mkdir dir="${bin.main.dir}" /> <javac destdir="${bin.main.dir}" debug="on"> <src path="${src.main.dir}" /> </javac> <copy todir="${bin.main.dir}"> <fileset dir="${src.main.dir}"> <exclude name="**/*.java" /> </fileset> </copy> </target> <target name="compile-src.test"> <mkdir dir="${bin.test.dir}" /> <javac destdir="${bin.test.dir}" debug="on"> <src path="${src.test.dir}" /> <classpath location="${bin.main.dir}" /> </javac> <copy todir="${bin.test.dir}"> <fileset dir="${src.test.dir}"> <exclude name="**/*.java" /> </fileset> </copy> </target>

Compilation is divided into two phases: first compile the source code, and then compile the test case code.

Listing 4 insert bytecode into the code of the class to be tested

<! -Inject bytecode to the Java class compiled in the path bin. Main. dir, And store the new Java class injected with bytecode to the path bin. instrument. dir --> <! -Metadata of coverage is stored in the path coverage. dir --> <Target name = "instrument"> <Mkdir dir = "$ {bin. instrument. dir}"/> <Mkdir dir = "$ {coverage. dir}"/> <Emma enabled = "$ {Emma. enabled}"> <Instr release pathref = "classpath. Main" Destdir = "$ {bin. instrument. dir }" Metadatafile = "$ {coverage. dir}/metadata. Emma" Merge = "true"> </Instr> </Emma> <Copy todir = "$ {bin. instrument. dir}"> <Fileset dir = "$ {bin. Main. dir}"> <Exclude name = "**/*. Java"/> </Fileset> </Copy> </Target>

When $ {Emma. enabled} is true, the class for inserting bytecode is generated. <Instr> specifies the address of the class to be instruments, the address to store the class after instruments, and the address to store metadata.

Listing 5 run the test case to obtain the metadata of some reports

<! -Generate the JUnit test report and Emma code coverage report when executing test cases --> <Target name = "test"> <Mkdir dir = "$ {junitreport. dir}"/> <JUnit fork = "true" forkmode = "once" Printsummary = "withoutandr" Errorproperty = "test. error" Showoutput = "on"> <! -Specify the location where metadata of code coverage is stored --> <Jvmarg Value = "-demma. Coverage. Out. File =$ {coverage. dir}/metadata. Emma"/> <Jvmarg value = "-demma. Coverage. Out. Merge = true"/> <Classpath location = "$ {bin. instrument. dir}"/> <Classpath location = "$ {bin. Test. dir}"/> <Classpath refID = "Emma. lib"/> <Formatter type = "XML"/> <! -Execute all JUnit test cases ending with test --> <Batchtest todir = "$ {junitreport. dir}" haltonfailure = "no"> <Fileset dir = "$ {bin. Test. dir}"> <Include name = "**/* test. Class"/> </Fileset> </Batchtest> </JUnit> </Target>

Set jvmarg before running the test case. All test cases run on the instrumented class.

Listing 6 generate a JUnit report

<Target name = "gen-Report-JUnit"> <! -Generate JUnit test report --> <Junitreport todir = "$ {junitreport. dir}"> <Fileset dir = "$ {junitreport. dir}"> <Include name = "*"/> </Fileset> <Report format = "frames" todir = "$ {junitreport. dir}"/> </Junitreport> </Target>

Listing 7 generate a coverage report

<! -Generate code coverage report --> <Target name = "gen-Report-coverage"> <! -If the value of Emma. enabled is true, a code coverage report is generated. --> <Emma enabled = "$ {Emma. enabled}"> <Report sourcepath = "$ {SRC. Main. dir }" Sort = "+ block, + name, + method, + class" Metrics = "method: 70, block: 80, line: 80, class: 100"> <Fileset dir = "$ {coverage. dir}"> <Include name = "*. Emma"/> </Fileset> <Html outfile = "$ {coverage. dir}/coverage.html" Depth = "method" columns = "name, class, method, block, line"/> </Report> </Emma> </Target>

<Report> sourcepath specifies the location of the source code to show the coverage of each line of code. Sort indicates the order of the generated list. "+" indicates ascending, and "-" indicates descending. Metrics can specify a coverage threshold for each measurement. If this threshold is not reached, the row is marked (provided that this function is supported in the form of reports, such as HTML ). <HTML> Generate a report in HTML format, depth indicates the details of the report, and columns indicates the order of column names in the generated list.

Back to Top

Show report

Now that we have written the ant script, you can run it. Assume that you have set up the environment for running ant and JUnit, directly go to the directory where the script is located, and enter ant in the command line.

The reports at various levels are as follows:

Figure 2 project-level reports

Figure 3 package-level reports

Figure 4 hierarchical reports

Figure 5 Source Code marked with color

You will find three colors: green, red, and yellow. They indicate the row: tested, not tested, and partially tested. The red or yellow part is worth your attention, and the bug may be hidden in this part of the code. What you need to do is to design some test cases, run the statements that were not previously executed. As shown in the figure above, some information is provided. If the string contains the "+" number, it is not tested, and "ispositive" is only tested to true or false, you need to add some test cases accordingly. Run the newly added test cases. You may find new bugs and fix them.

Back to Top

Problems hidden behind reports

For this simple example, you will find that we can easily reach 100% of the test coverage rate. You may feel relieved to say: Ah, I have tested all the cases. We are sorry to tell you that Emma has limited functions and does not support decision coverage and path coverage. In fact, an exhaustive test of a slightly complex project is impossible.

Listing 8 sample code for decision override and path override

/** * Parses the given string to a float number * * @param number * the given string * @return the float number related with the string * * @throws IllegalArgumentException * if the string is empty, null or can not parse to a float */ public float parse(String number) { if (number.equals("")||number == null ) { throw new IllegalArgumentException( "Number string should not be empty or null"); } StringIterator stringIterator = new StringIterator(number); getSign(stringIterator); int integer = getInteger(stringIterator); float fraction = getFraction(stringIterator); float total = integer + fraction; return isPositive ? total : (-1) * total; }

Listing 9 test cases of decision coverage and path coverage

public void test_parse () { NumberParser np = new NumberParser(); String number =""; try { np.parse(number); fail("should throw IAE"); } catch (IllegalArgumentException e) { // pass } number = "22.010"; float parsedNumber = np.parse(number); assertEquals((float) 22.010, parsedNumber); number = "-22.010"; parsedNumber = np.parse(number); assertEquals((float) 22.010, parsedNumber); }

Run the ant script to generate the report. You will find that all the test cases have passed and the test coverage report shows that all the lines of the Code have been executed. However, careful readers must have seen bugs in the above Code. If the string passed into parse is null, it does not get illegalargumentexception as we wish, but throws nullpointerexception.

Although the line below is green, it only indicates that each condition statement has been executed, and does not indicate that each condition has both true and false. In the test case we designed, "null = Number" only gets false. We need to add a test in our test case where the string condition is null.

Figure 6 Decision coverage and path coverage report

Listing 10 Code bug fixes

if (null == number || "".equals(number)) {

Back to Top

Conclusion

Emma is a good choice for generating coverage reports for your project. Through the coverage report, we can discover and fix some hidden bugs, and our software will become stronger.

From IBM: http://www-128.ibm.com/developerworks/cn/java/j-lo-emma/index.html