Code coverage analysis

Whether it is unit testing, API testing, or functional testing, the code of the product is eventually called. How to evaluate the efficiency of these tests and whether all or most of the code of the product is covered? At this time, code coverage is a valuable reference indicator.

Generally, code coverage is used in the following aspects:

• Find out the places where the code is not tested;
• Add new test code to improve code coverage;
• Analyze the efficiency of the test code to design more effective test code or test cases.

Common metrics of code coverage

Statement overwrite (statement coverage ):Statement overwrite refers to whether each line of code of the program is overwritten. Statement overwrite is the most commonly used code coverage indicator, which is also very simple. But for some code that controls the structure, it cannot really indicate whether it is completely covered, as shown in the following code:

int* p=NULL;if(condition)    p=&variable;*p=123;

In the preceding code, if condition is true, the statement coverage rate is 100%. In fact, when the condtion is false, it will lead to a runtime error in which the NULL pointer is assigned a value.

 

Decision coverage ):Decision overwrite refers to the code block that controls the structure, for example, whether the entire bool expression in IF and while is executed once under the conditions of false and true. Decision coverage does not consider whether the subconditions that constitute the condition expression are completely overwritten. Therefore, for C/C ++/Java, because short circuit operators is a conditional expression consisting of multiple conditions and relations, when the first condition is detected as false, all other conditions will not be executed. Consider the Code:

if(condition1 && (condition2 || function1()))    statement1;else    statement2;

When condtion1 is false, conditon1 is true, and condition2 is true, the decision coverage rate is 100%, while the code in function1 is never executed. Decision coverage is also called branch coverage ).

Conditional coverage ):Conditional overwriting means that each condition must have both true and false. Here, a condition is the operand of a logical operator, but does not include this logical operator. The difference between attention and decision coverage, such

bool f(bool e){    return false;    }bool a[2]={false, false};if(f(a&&b)) ...if(a[int(a&&b)]) ...if((a&&b)?false:false) ...

The three if conditions in the code block contain two conditions A and B. In four cases, A = true, A = false, B = true, and B = false; we only need to take the combination of true and false for A and B (A = true, B = false and a = false, B = true) to overwrite the condition to 100%. In fact, no matter what value A and B take, the branches of the three if statements always return false.

Multiple condition coverage ):Multi-condition coverage indicates whether each possible combination of conditions is checked. The combination of conditions in Multi-condition coverage is an arrangement, not a combination. Generally, the design of multi-condition coverage test cases is very complex, because when considering the arrangement relationship between each condition, the number itself is large, and the mutual influence between the relations must also be considered, remove invalid sorting.

 

Condition/decision coverage ):Conditional decision coverage is a joint operation between conditional coverage and decision coverage. It has advantages over multi-condition coverage, and does not have conditional coverage or decision coverage.

 

Improved condition/decision coverage (modified condition/demo-coverage ):It is defined that every entry and exit in a program is called at least once, every condition of a decision has at least one output of all possibilities, and every decision in a program has at least one output of all possibilities, each condition in a decision can independently influence the output of the decision (the condition changes while other conditions remain unchanged ).

 

Path coverage ):Path decision indicates whether each possible path in the function is executed. A path is a unique branch sequence from the function entry to the exit. Path coverage can usually be thoroughly tested, but it also has two very serious defects: first, the number of paths is the geometric series of the number of branches. For example, a function with 10 if statements needs to be tested in 1024 paths. If we add an if statement, 2048 paths need to be tested. Secondly, due to the relationship between data, some paths cannot be tested, such:

if(success)    statement1;statement2;if(success)    statement3;

This Code contains four paths. In fact, there are only two feasible items, success = false and success = true.

Other metrics of code coverage

Other code coverage indicators include function coverage, call coverage, and loop coverage.

Relationship between code coverage

• Decision overwrite is usually covered by statements, because the execution of each branch requires the execution of each statement. However, if the control flow of a function is interrupted by statements such as throw and abort, this rule does not apply.
• Condition/decision coverage includes condition coverage and decision coverage.
• Path coverage includes decision coverage.

How to use code coverage data

We usually want to design efficient test cases with less time. Simply put, an efficient test case is to discover as many problems as possible in the program. The time, coverage, and detected time of the test can be found in the following chart:

Even if the code coverage rate is 100%, we can only say that each line of code in the program has been executed. However, if we must have a quantitative indicator, see the following table:

Coverage type	Without dependency Injection	With dependency Injection
Function	90%	99%
Line	75%	95%

Ref: http://blogs.msdn.com/ B /cellfish/archive/2008/06/16/code-coverage.aspx

Dependency injection (DI) is a design mode, that is, to introduce an object into a class, rather than relying on the class to create the object itself. Specific reference: http://msdn.microsoft.com/en-us/magazine/cc163739.aspx

It is worth noting that:

• Do not blindly pursue data and design test cases to improve code test coverage;
• The Code test coverage rate should be used as a reference to help us design more meaningful and efficient test cases;

If you or your boss must have a very beautiful number, or you use it to consider the performance of testers, then. for programs on the net platform, Microsoft has a tool called PEX that can help you generate test cases with high code coverage. For details, refer:

Http://research.microsoft.com/en-us/projects/pex/

 

Note: For most of this article, refer to code coverage analysis. You are welcome to reprint it, but please indicate the source. Thank you.