Software Testing defect Metrics analysis

Measurement of defects is helpful for testing process monitoring such as defect density analysis, number of defects found and repaired, etc. In addition, the defect metrics should include the defect information needed to track the process of process control information improvement activities, and introduce defect source analysis, defect trend analysis, etc. as input to the risk mitigation strategy. This paper introduces several common defect metrics, in the actual project, the defect metric is usually used in conjunction with other indicators to achieve the purpose of measurement.

1. Defect Discovery Progress

1) Measure the target

Defect Discovery Progress measures (cumulative defects) can show the cumulative number of defects found per week, helping to assess the status of the test, the progress of the test, and the quality of the software system.

2) Metric definition

The x-axis of the Defect Discovery Progress measure (cumulative defect) is the week, expressed in yww form, where Y represents the last two digits of the year, and WW represents the week, for example: 815 refers to the 15th week of 2008. The y-axis represents the number of defects found during the testing phase, as shown in 1.

Figure 1 Defect Discovery Progress (cumulative defect)

3) Measurement Analysis

When analyzing the defect discovery progress, we can evaluate the test status, test progress and test quality, as well as the allocation of subsequent test resources from the following aspects:

Combine defect repair progress metrics to analyze differences in the number of defects found and repaired, and help project teams make rational resource allocations from the overall project level.

Analyze the peak time of defect discovery, that is, the time when the defect discovery progress curve begins to flatten out, if the defect finds that the time point of the progress curve is far away from the end of the planned test, the reason should be analyzed.

Combine with other metrics such as test case execution progress, analyze defect discovery progress to meet test quality, test progress requirements, and analyze the cause.

2. Defect Repair Progress

1) Measure the target

Defect Repair Progress measures (cumulative defects) can show the cumulative number of bug fixes per week, helping to assess the developer's progress in fixing defects, evaluating subsequent test resource allocations, and the quality of the software system.

2) Metric definition

The x-axis of the defect repair progress (cumulative defect) is time, expressed in yww form, where Y represents the last two digits of the year, and WW represents the week, for example: 815 refers to the 15th week of 2008. The y-axis represents the number of defects found during the testing phase, as shown in 2.

3) Measurement Analysis

When analyzing the progress of defect repair, it is possible to analyze and evaluate the progress of the developer to fix the defect, evaluate the subsequent test resource allocation and the quality of the software system from the following aspects:

Combine defect discovery Progress metrics to analyze differences between the number of defects found and fixed defects, and help project teams make rational resource allocations from the overall project level.

According to the test defect repair Progress metrics data, analyze the developers to fix the rate of defects within the normal range, and analyze the causes of large differences.

The allocation of test plans and test resources can be updated in a targeted manner, depending on the developer's bug fixes.

Figure 2 Bug fix progress (cumulative defect)

3. Defect Priority

1) Measure the target

Defect priority measurement (cumulative defect) helps to identify the proportion of defects of different priority levels in all defects, and to grasp the distribution of different priority defects on a holistic basis, helping to develop and test the planning and distribution of resources.

2) Metric definition

The x-axis of the defect priority measure (cumulative defect) is time, expressed in yww form, where Y represents the last two digits of the year, and WW represents the week, for example: 815 refers to the 15th week of 2008. The y-axis indicates the number of tests, as shown in 3. The test data that needs to be collected includes:

The number of defect priorities is 1 (cumulative defect).

The number of defect priorities is 2 (cumulative defect).

The number of defect priorities is 3 (cumulative defect).

The number of defect priorities is 4 (cumulative defect).

Figure 3 Defect priority (cumulative defect)

3) Measurement Analysis

The test progress and test quality can be analyzed in the following ways when the defect priority metric is analyzed:

The number of defects found for each priority level.

There is no sudden big change in Priority 1 and Priority 2 defects, and high-priority defects often mean a large increase in product quality.

and defect Detection progress measurement (cumulative defect) data, analysis of each priority defect detection trends, to determine whether the product quality is stabilized.

4. Severity of defects

1) Measure the target

The defect severity measure (cumulative defect) helps to identify the proportion of defects in all defects in different severity levels, grasps the distribution of different severity defects in the whole, helps to plan and distribute the developer resources, as well as the planning and distribution of the testers resources.

2) Metric definition

The x-axis of the defect severity measure (cumulative defect) is the week, expressed in yww form, where Y represents the last two digits of the year, and WW represents the week, for example: 815 refers to the 15th week of 2008. The y-axis indicates the number of tests, as shown in 4. The test data that needs to be collected includes:

The number of defect severity is 1 (cumulative defect).

The number of defect severity is 2 (cumulative defect).

The number of defect severity is 3 (cumulative defect).

The number of defect severity is 4 (cumulative defect).

Figure 4 Defect Severity (cumulative defect)

3) Measurement Analysis

The test progress and test quality can be analyzed in the following ways when the defect severity measurement is analyzed:

Number of defects found in severity of each defect.

Severity 1 and Severity 2 defects have no sudden big changes, high severity of the defect mass increase usually means that the product quality problems.

and defect Detection progress measurement (cumulative defect), the analysis of the detection trend of each severity defect, to determine whether the product quality is stable.

Reprint: http://www.uml.org.cn/Test/201405233.asp

Software Testing defect Metrics analysis