Code coverage from simple to complex (a)

Code coverage is an important indicator of how well a test case covers the software under test and is an important indicator for measuring the progress of testing. It is also one of the important indicators to quantify the testing effort, which is often less exciting than development, one of the important reasons is that testing is difficult to quantify, and code coverage is precisely the key to solving a problem. According to the content of the different, can be subdivided into: statement coverage, decision coverage, conditional coverage, path coverage and loop coverage, and so on, here is a very good blog "code Coverage" introduced a variety of different coverage definitions. Some understand it is quite awkward, but in fact, it is not difficult to use to see it! In all of these overrides the statement overlay (Statement coverage) is the simplest and most commonly used, the most effective coverage, and Visual Studio uses the basic block coverage in the statement overlay.

For Agile development Teams, code coverage is one of the hard quality standards (Exit criteria) to be completed for each sprint, and coverage varies depending on the project: 75%,80% or even 100% is possible. Code coverage is a white-box concept, after all, it's finally going to be implemented into code. Since code coverage is so important, when should you use it? How do I use it?

Some people think that code coverage is important, so from the beginning of the project Code coverage inspection and analysis, that is, get coverage –> found not covered code –> add new test cases. In this way, I named it " Code Coverage driven test (Ccdt,code coverage driven test)". CCDT looks beautiful, theoretically invulnerable, but the actual operation is completely not the case. First not to say whether this way is correct, the cost of the introduction of this is enough for the project team to drink a pot of, hehe! That's because CCDT needs to go through the three steps of getting coverage, analyzing coverage, and adding test cases, each with a lot of potential "by-products" overhead, especially in the first two steps. To get coverage, you need to execute all the test cases, and you know that 70% of the industry's tests are still manual, and it's obviously unrealistic to run test cases frequently for coverage only, and frequent analysis of coverage results is a time-consuming task, both for developers and testers. Especially for teams adopting agile development methodologies, short iterations do not allow the introduction of such laborious work. Hu in his blog that the only coverage symptom is "test coverage obsessive-compulsive disorder", which has a very good description:

"Test coverage can only tell the team what hasn't been tested, and it doesn't answer whether the software has been tested effectively !" ”

I think for the Test team, the test process should be the User Scenario Overlay driver Test (Uscdt,user Scenario coverage driven test) , that is, the tester should be from the user's actual use of the scene, Think about what you want to test and design your test cases. Code coverage is a necessary complement to USCDT to discover scenes that are not covered (Test Hole). Code coverage should be introduced in the middle and late phases of the project/iteration, not very frequently, donuts, for example, for a 3-4-week iteration, and 3 coverage checks are sufficient. Of course, if you have a high proportion of automated tests, using a continuous integration (continuous integration) approach, such as the build feature of continuous integration in Visual Studio 2010, you can automate code coverage statistics every time, or earlier, Code coverage is more frequent, but only if it does not cause excessive overhead.

With all that said, here's a little more practical, see how to get Code coverage data in Visual Studio 2010! The Code coverage data for automated test cases in Visual Studio's integrated development environment is the simplest (slightly more complex with a command line approach, but it can also be used to obtain overwrite data for manual test cases), which requires only the following three steps:

Step One : Select the code Coverage in the test settings configuration

(Updated 2011/6/1) It is important to note that when you select the code coverage item, you also select the "Configure" button to configure which file (. exe/.dl) to collect coverage data for, Visual Studio will automatically instrument the file, otherwise it will not collect the overwritten data.

Step Two: Execute automated test Cases

Step three : View Code Coverage results

With the modernization and fool of programming languages and development tools, it is becoming easier to get and analyze code coverage now, but the use of code coverage data remains in a relatively primitive phase. There are plenty of great things to dig into to help test teams optimize the testing process and the design of test cases. "Code coverage from simple to complex (iv) – for code check-in" is described in the article, how to introduce a coverage check in the code check-in process.

Code coverage from simple to complex (a)