Test the coverage of statements, branches, and paths in Java

Introduction

CodeOverwriting is a method used to measure the level of executed software tests. The process of collecting covered metric data is simple: monitor your code and run the test on the version you are monitoring. In this way, you can generate relevant data to show which code has been executed, or, more importantly, which code has not been executed. Coverage testing is a perfect complement to unit testing: unit testing can show whether the code is executed as expected, while code coverage can indicate the code to be tested.

Most developers can understand this process and agree with its value proposition. They generally pursue a 100% coverage rate. Although 100% of coverage is an excellent target, improper 100% coverage remains unknown. Typical software development measures coverage based on the number of statements or branches to be tested. Even with 100% statement or branch coverage, the Code logic may still have serious logic bugs, which can only bring false security to developers and administrators.

Why is 100% coverage insufficient? This is because the statement and branch coverage cannot indicate whether the logic in the code has been executed. Statement and branch coverage are useful for obvious problems in unexecuted code blocks, but they often miss bugs related to decision structures and decision interaction. On the other hand, path coverage is a more robust and comprehensive technology that helps to detect defects early.

Before learning about path coverage, pay attention to the statements and branch coverage:

Statement Overwrite

Statement overwrite can identify the statements executed in a method or class. This is a simple algorithm, and many open-source products can evaluate this coverage level. In the end, the benefit of statement coverage is that it can recognize unexecuted code blocks. However, statement overwrite also has problems and cannot be identified.Source codeBugs caused by the Control Flow Structure in, such as composite conditions or continuous switch labels. This means that while you can easily get 100% coverage, no significant bugs are found.

This problem is shown in the following example. Here, the returninput () method is composed of seven statements. It has a simple requirement that the output should be equal to the input.

Figure 1. Sample Code

Next, you can create a JUnit test case that meets your needs and has 100% statement coverage.

Figure 2. Statement Overwrite

There is an obvious bug in returninput. If the calculation of the first or second decision is true and the calculation of other statements is false, the return value is not equal to the input value of this method. A savvy software developer immediately notices this problem, but the statement coverage report shows 100% coverage. If the Administrator finds that the coverage rate is 100%, he/she may be affected by a false sense of security, determine that the test has been completed, and then release code with hundreds of errors to put it into production.

It is not enough to understand statement coverage. Developers must further use a more sophisticated testing technology: branch coverage.

Branch coverage

The branch refers to the decision result. Therefore, the branch coverage can evaluate the tested decision result. This sounds good, because it can view the source code more deeply than the statement overwrite, but the branch Overwrite will also put forward more requirements.

It is very easy to determine the number of branches in the method. Boolean decisions undoubtedly have only two results: true and false, and the switch has only one result for each case-Do not forget the default situation! The total number of decision results in the method is equivalent to the sum of the branches to be covered and the input branches in the method. (After all, the method of using linear code also has a branch ).

In the preceding example, returninput () has seven branches-three are true, three are false, and the other is hidden Branch used for method input. You can use two test cases to cover six true and false branches:

Figure 3. branch coverage

Both tests validate the (output equals to input) requirement and reach 100% of the branch coverage rate. However, no bugs were found in the test despite the 100% branch coverage. The Administrator may think that the test has been completed again and this method can enter the production stage.

Smart developers will realize that you may miss some possible paths in the tested method. The preceding example has tested the true-False-True or false-True-true path. You can add two additional tests for the test.

This method has only three decisions, so it is very easy to test all eight possible paths. However, for more decision-making methods, the number of paths may increase exponentially. For example, a method that contains ten Boolean decisions has 1024 possible paths. Good luck!

Therefore, obtaining 100% of statements and 100% of branch coverage is far from enough, but it is hard to test all possible paths contained in a complex method. Are there any other options? Let's take a look at the basic path coverage.

Basic path Overwrite

The path indicates the execution process from the execution method to the exit method. A method with n decisions has 2 ^ n possible paths. If a method contains a loop, an infinite number of paths are generated. Fortunately, you can use a measurement called cyclomatic complexity to reduce the number of paths to be tested.

The complexity of a method is the number 1 plus the number of unique decisions in the method. Circle complexity helps you define the number of linear independent paths for a method, that is, the base group. The definition of linear independence is not covered in this article, but in general, the base group is the smallest path group, which can be used together to create other possible paths in the method.

Like branch coverage, the path base group test ensures that you can test each decision result. However, unlike branch coverage, basic path coverage ensures that all mutually independent decision results are tested. That is to say, each new basic path can accurately "Browse" the previous execution decision without changing other execution branches. This is an important reason why basic path coverage is more robust than branch coverage. It enables you to understand how to change a decision to influence the behavior of a method.

I will use the same example for illustration.

Figure 4. Sample Code

To get 100% of the basic path overwrites, You need to define the base group. The complexity of this method is four (1 plus the number of decision-making), and four linear independent paths need to be defined. To this end, you can set any first path as a baseline and browse the decision one by one until the base group is completed.

Path 1: Any path can be used as a baseline, so the decision result is selected as true (represented by TTT ). This is the first path in the base group.

Path 2: to find the next basic path, browse (only) the first decision in the baseline path and display FTT to indicate the expected decision result.

Path 3: The second decision in the browsing baseline path. The third basic path is TFT. In this way, the first baseline decision can remain "true" as the result.

Path 4: Finally, the third decision in the baseline path is viewed, and the fourth basic path is TTF. In this way, the first baseline decision can remain "true" as the result.

Therefore, the four basic paths are TTT, FTT, TFT, and TTF. Now, test and view the results.

In the attached code, you can see that testreturninputintbooleanbooleanbooleantft () and testreturninputintbooleanbooleanbooleanftt () locate bugs not found in statement and branch overwrite. In addition, the number of basic paths increases linearly with the decision data, rather than exponentially, so that the number of tests required is equal to the number required to achieve full branch coverage. In fact, the basic path test overwrites all the statements and branches in a method, so it effectively contains the branch and statement overwrites.

But why not test other potential paths? Remember, the basic path test aims to test all relatively independent decision results. This can be achieved through testing the four basic paths, which has nothing to do with other paths. If you select fff as the baseline path at the beginning, you will get the base group (FFF, TFF, FTF, and FFT), regardless of the TTT path. Both base groups are equally valid, and any one can meet your independent decision-making result standards.

Create Test Data

In this example, it is very easy to get 100% of the basic path coverage rate, but a comprehensive test of the path base group in real life will be more challenging, or even impossible to achieve. Because the basic path overwrites the interaction between decisions in a method, you need to use the test data that promotes execution in a specified path instead of simply using a single decision result, this is also necessary for branch coverage. It is very difficult to forcibly execute by injecting data to a specified path, but you can remember several coding practices that simplify the test process.

1. Keep your code simple. The lower the lap complexity, the better. Refactor when the value can be reduced. The higher the value, the more difficult it is to test. Once the value exceeds 10, the test cannot be implemented. This not only reduces the number of basic paths to be tested, but also reduces the number of decisions in each path.
2. Avoid repeated decisions
3. Avoid data dependency

Consider the following example:

Figure 5. Test the basic path

The X variable is indirectly dependent on the object1 parameter, but the inserted code makes it especially difficult to view the relationship. As methods become more complex, it is almost impossible to view the relationship between input values of methods and decision expressions.

Conclusion

Although the statement and branch coverage values are easy to calculate and obtain, they may leave unknown major defects and can only bring false security to developers and administrators. Basic path coverage provides a more robust and comprehensive approach to discover these imperceptible defects and avoid exponential growth in the number of tests required.