Code coverage from simplicity to complexity (1)

Code coverage is an important indicator that reflects the coverage of test cases to tested software. It is also an important indicator to measure the testing progress. It is also an important indicator for Testing. Testing is often not as exciting as development. One of the important reasons is that testing is difficult to quantify, code coverage is an important indicator to solve a problem.

It can be subdivided into statement overwrite, decision overwrite, condition overwrite, path overwrite, and loop overwrite based on its coverage, here is a good blog about code coverage, which introduces various definitions of different coverage rates. Some of them are quite easy to understand, but it is not hard to understand! In all these overwrites, statement overwrite (statement coverage) is the simplest, but also the most commonly used and most practical and effective coverage, visual Studio uses basic block coverage ).

For the agile development team, code coverage is one of the hard quality standards (exit criteria) to be completed by each sprint. The coverage varies by project: 75%, 80% or even 100% are possible. Code coverage is a white box concept. It should be implemented to the code after all. Since code coverage is so important, when should I use it? How can I use it?

Some people think that code coverage is important, so code coverage should be checked and analyzed from the beginning of the project, that is, getting coverage-> discovering unoverwritten code-> adding new test cases. In this way, I name it"Code coverage rate-Driven Testing (ccdt, code coverage driven test)". The ccdt looks pretty and theoretically impeccable, but the actual operation is completely different. Not to mention whether this method is correct. For the overhead introduced by this method alone, it is enough for the project team to have a pot of drinks! This is because ccdt requires three steps: Getting coverage, analyzing coverage, and adding test cases. Each step has many potential "by-product" overhead, especially in the first two steps. To obtain coverage, all test cases need to be executed. You know that 70% of the tests in the industry are still manual. It is obviously unrealistic to execute test cases frequently only for coverage; frequent Analysis of coverage results is also a time-consuming task, whether it is done by developers or testers, especially for teams that Adopt agile development methods, short iterations do not allow the introduction of such laborious work content. In his blog, Hu Kai described the symptoms of this coverage-only theory as "test coverage obsessive-compulsive disorder". One of the descriptions is wonderful:

Test coverage can only tell the team whatNoIf the software is tested, it cannot be answered whether the software has passed the test.ValidTest!

I think the testing process should beUser scenario coverage driver test (uscdt, user scenario coverage driven test)That is, testers should consider the content to be tested and design test cases based on the actual use cases of users. Code coverage is for uscdtNecessary supplementTo discover the scenarios that are not covered (test hole ). Code coverage should be introduced in the middle and late stages of the project/iteration, and should not be frequent. For example, for a 3-4 week iteration, three coverage checks are enough. Of course, if your automated testing ratio is relatively high, you can use the continuous integration (continuous integration) method, for example, the build function of continuous integration in Visual Studio 2010, the code coverage rate can be calculated automatically every time, and the code coverage rate can be performed earlier and more frequently, but the premise is that this will not bring too much overhead.

After talking about this, let's take a look at how to get code coverage data in Visual Studio 2010! In the integrated development environment of Visual Studio, obtaining the code coverage data of automated test cases is the simplest (using the command line method is slightly more complex, however, it can also be used to obtain overwrite data for manual test cases. The following three steps are required:

 

Step 1: Select code coverage in test settings configuration

 

 

(Updated 2011/6/1)Note that when you select a code coverageitem, you also need to select the file (.exe/. dl) to which the program wants to collect Overwriting data. Visual Studio will automatically perform the instrument operation on the file, otherwise it will not collect Overwriting data.

 

Step 2: Execute automated test cases

 

 

Step 3: view the code coverage result

 

As programming languages and development tools are modernized and foolish, it is now easier to get and analyze code coverage. However, the use of code coverage data remains in the original stage. There are also many good things worth exploring in depth to help the test team optimize the test process and test case design!

References

1. Code coverage Instrumentation
2. How to: Configure code coverage using test settings for automatic tests
3. Wikipedia's code coverage topic
4. Team system "off-road" code coverage analysis and reporting experience