Quality evaluation guideMake by Tony 2006/11 in the test process we usually use the testing quality evaluation method mainly test coverage, where the test coverage mainly includes statement coverage, decision coverage (also known as branch coverage), condition coverage, decision/condition coverage, and condition combination coverage. However, according to our long-term practical experience, it is far from enough to cover only these standards. There are still many vulnerabilities, leading to "smart" testers being lazy, however, the Quality Assurance personnel and test managers of a company are pessimistic about this situation. Recently, I used a test tool vctester to find that the evaluation method of this tool is somewhat advanced. At least in the current status, it can fill in many assessment vulnerabilities, which has aroused my constant reflection, here, I also want to explain it to you. The Quality Evaluation System of vctester consists of two parts: Test Coverage Evaluation and test design evaluation, in addition, it also includes a standard custom mechanism for coverage, which is described below.
I,
Test Coverage EvaluationSupports two types of Test Coverage Evaluation: location-independent call coverage and licc, and location-dependent call coverage and ldcc ), call overwrite is actually a successful application in the Java field. It is defined as follows: licc = (number of non-repeated function calls already covered/number of all non-repeated function calls) * 100% ldcc = (number of function calls covered/number of all function calls) * 100% for example, a function calls three subfunctions, among them, 1st subfunctions appear in the two defined parts of the function, and the other two subfunctions are called only in one place. If all three sub-functions have been called, 1st of them are called only at one position, and the other is not yet called. In this case, we calculate that licc is "3/3 = 100%", and ldcc is "3/4 = 75% ". Licc indicators are mainly used for rough tests to ensure a certain degree of initial stability of a module and are suitable for integration with other modules. licc indicators try to explain whether the newly defined functions have been used, it doesn't matter whether everything is running. Ldcc is used in the formal test to fully evaluate the test degree.
II,
Test Design EvaluationTest Case coverage (TCC), also known as test case coverage, is designed based on the relationship between the frequency of use of the tested function in the formal test and the total number of branches in the function definition. It is defined as follows: TCC = Total number of calls to the function under test/total number of branches defined by the function. The total number of branches of a function is defined as follows: total number of function branches = 1 + total number of IF Statements * 2 + total number of while statements * 2 + total number of for statements * 2. If statements have two branches: True and Fals, in principle, two tests should be designed to be complete. While and for are similar. At least one test case should be designed for a loop, and one test case should be designed for a loop, therefore, after calculating the total number of if, while, and for statements, multiply the number by 2. But if a function does not use these three control statements, at least one test case should be designed to test it, this is the additional meaning of 1 in the above formula. Test Case coverage is usually not a hard evaluation indicator of test results, but only a lower limit is set. If the test result fails to reach the standard, it should be rejected by one vote. You can set this indicator to monitor casual tests. That is, code coverage increases when you call a function that is not tested, leading to a high coverage.
III,
Standard for custom coverageDuring the test, the "exception list" can be defined for functions that are not concerned with the test and error handling functions that do not want to be noise in the test evaluation ", in addition, licc, ldcc, and TCC evaluation coefficients can be customized. This evaluation system brings great flexibility and practicality to the evaluation activities.
Appendix: Introduction to vctesterVctester
Shenzhen lingzhou Technology Co., Ltd.Self-developed, professional testing tool for Embedded white box testing, which follows the 4th-generation white box testing method (4gwm, the 4th generation white-box-testing methodology ), to effectively implement unit testing, integration testing, and protocol testing for C language, a systematic testing solution is provided. For more information about vctester, see its official website http://www.eztester.com.
References:1. wayne Chan, Overview of the 4th-generation white box testing method-theoretical 2. wayne Chan, 4th white box testing methods overview-vctester Practice 3. philip M. johnson, and Joy M. agustin, "Keeping the coverage Green: investigating the cost and quality of testing in Agile development" Welcome to repost this article, please indicate the source of the article when reprinting: vince Testing Technology Research Center http://blog.csdn.net/vincetest I like the site: Sina Google News Sohu news Yahoo day csdn news csdn Blog B weekly blog center donews blog sourceforgeshareit carefree test VC online to create the Alliance huajun Software Park
