Using contest to analyze code coverage for test cases

Solve what problem

The problem is presented as follows: The first problem for testers is the inability to measure the quality of test cases, and if test engineers spend a lot of time writing test cases that do not effectively cover important implementation code, it can be shown that such test cases are not good. At the same time, you can analyze why you did not overwrite the important code according to the reports covered by the test, and then need to improve the code coverage of the test cases to achieve satisfactory results. Code coverage varies depending on the product: 70%,80% or even 100% is possible. For test engineers, you can follow this process: Get coverage –> find uncovered code –> add new test cases to improve test cases to ensure product quality.

Basic introduction to Code Coverage

Four metrics were defined in the testing process of the CMMI4 system: Code coverage, test execution rate, test execution pass rate, and test defect resolution rate. The code coverage is a description of the program's source code in the functional testing coverage, that reflects the test case of the software to be tested to achieve the level of code coverage of the important indicators, it is also an important indicator of the quantification of the test work. Test work is often less likely to be developed than to represent results with chart data, one of the important reasons is that testing is difficult to quantify, and code coverage is an important indicator to address this problem. At the same time, the risk of software is likely to exist in areas where code coverage is not covered, code coverage can be found in the uncovered scenario (test Hole), and test engineers analyze test cases based on this test Hole to optimize test cases. This process can significantly improve the production efficiency of the test and improve the quality of the SOFTWARE PRODUCT. Code coverage is the concept of a white-box test, after all, to trace the final source code. In general, code coverage testing should be introduced in the middle and late stages of a product, but for products developed using agile technologies, they can be introduced at the beginning of the iterative cycle of development. The author's team is used in the middle and late stages of the project, primarily for testing the code coverage of regression tests.

Code Coverage Basic Metrics

Statement Overlay Statement Coverage (line coverage): Used to evaluate the row coverage performed by a product code line in a test activity, the advantage is simple, and the disadvantage is that it cannot contain logical branches of code.

Basic block coverage: the non-branched code area is used as a unit of measurement, and for one branch of a if/else the number of rows is much larger than the other, so there may be a higher case for this indicator.

Conditional coverage Decision Coverage (branch coverage): used to evaluate code coverage for code branching.

Feature overrides function coverage (method function): Evaluates the coverage of the methods that the product code performs in the test activity.

How code coverage is implemented

For Java as a development language products, the analysis of code coverage when the use of instrumentation technology, the essence of the product code in the key position of the insertion of statistical code, that is, in the beginning of each method of the code to add statistical counters and serial numbers, when the method is executed, The test case is tested using the serial number to calculate the modification method. Generally speaking, instrumentation technology can be divided into two ways: Class instrumentation and Source instrumentation. The former inserts the statistic code into the compiled. class file, while the latter inserts the statistic code into the source code (. java file) and compiles it into a new. class file. Most code coverage tools use these two instrumentation techniques. The most widely used Class instrumentation, because in each phase of product development, the use of continuous integration to develop the product, the test engineer is not the product of the source code, but delivered to the test product version, so considering to avoid modifying the source code, and the direct injection of statistical code into the generated class, for this article also used the first class instrumentation technology.

Contest Introduction

Contest is a powerful easy-to-use tool that discovers the bugs of parallel systems in the early stages of testing without having to build complex testing environments for multiple processors and multiple applications. Contest is very helpful to developers of test engineers, either as stand-alone tools or as an Eclipse plug-in, where developers run to get code coverage for unit tests. Contest can measure code coverage with minimal cost, deadlock information can be found during debugging, Contest has many other features, and code coverage is a feature we use in this article. Interested readers can download the Contest here. Contest supports two levels of code coverage, covering method coverage and basic statement block Basic-block coverage.