Introduction to ODC (orthogonal Defect classification)--orthogonal defect classification

defect analysis is an important part of software development and testing, and ODC introduces a very effective defect classification and analysis method which is different from common people. This article simply introduces you to what is ODC and how to use ODC in project and product development to improve the development testing process to enhance product quality. Readers are expected to have basic software development and testing experience, and understand the basic methods of defect analysis.

Defect classification helps improve product quality

Software development includes a process for controlling software development. We design the module, develop the code, test the product, and then release the product. But how do we learn from our previous mistakes and how do we do better? In general, we will take some output data to analyze, so we know what we should do and what improvements (1). But how do we make sure that our efforts are really useful? This is where defect classification (defect classification) can help us, and if we can use defect classification correctly, it will help us a lot.

Figure 1

Back to top of page

Several common methods of defect classification

In the software development process we will find in different stages of the number of unequal defect,2, for the discovered defect we can analyze each of them, for example, using the root causal analysis method, but this method takes up a lot of time and resources, Obviously we really need a way to tell us exactly where we should improve.

Figure 2

Let's take a look at several of our common defect analysis methods:

Classification according to defect severity

We classify the defect according to the severity of the defect during the test, where we refer to the severity as severity, and we have a map of the defect of the different testing phases of the project shown:

Figure 3

In this diagram the defects is categorized according to their severity attributes, and the defect of severity 1 is the most serious defect, which makes the system simply incapable of functioning and requires immediate correction. That severity is 2 defect is a general functional error, these errors are required in the requirements, must be corrected in order to achieve the complete system function. Severity 3 defect are small errors that do not affect the implementation of the feature, but may cause misunderstanding or improper use of the user. The defect for severity 4 is where testers suggest improvements, and if time allows developers to make selective corrections or wait until the next release. From Figure 3 We can see that the first diagram is in a project before the test, when the level of 1 defect a lot, the whole system is not able to operate, it is required a lot of time and manpower to test and correct code errors. The second diagram shows that the project test is in the mid-term, when the worst defect are already scarce, the system is basically operational, and the testers have found a lot of functional errors and details of the errors that need to be corrected. The third figure shows that the project test has reached the end, when the function of the product requirements has been realized, only some details and recommendations need to be improved, the product can be released. In the chart with severity classification, we can learn about several aspects of the following projects:

1) Priority of work

2) Progress status of the project

3) Quality of the product

Classification by Component/module

For different component or module, we can also have similar defect map to illustrate some other problems:

Figure 4

In Figure 4, for the first diagram, we can see that the defect found in the C module is significantly less than the other modules, so the reason may be that the C Module developer technology is very good. In the second figure, we can see that the defect found in a and C is significantly more than the other two modules, so the difficulty, size, or change of the two modules can vary considerably, resulting in more defect found in them. For the third diagram, the C module defect significantly more than the other, then it may be the C module developers are too poor, need special attention of the manager.

Back to top of page

Orthogonal Defect Classification Introduction

Here we introduce ODC, what is ODC (orthogonal Defect classification)? Simply put, it is another way of defect classification, which allows you to quickly get information about each problem to help you make the right decision to solve the problem later.

Application of ODC in development

As a developer I have found a problem if categorized by type (type) may be by the following several possible: (the English in parentheses is the abbreviation legend)

1) No correct initialization (init)

2) The code does not have the correct check-in (CHK)

3) Algorithmic problem (ALG)

4) Functional error, it is possible that the function within the module is not implemented correctly, or that the part of the module that is connected to the module is not implemented correctly. (Fnct Cls)

5) There may be time-related errors

6) interface-related errors (Intf)

7) errors associated with the code, such as an incorrect inheritance relationship (Rel ' N)

According to the classification of type we have the following distribution map:

Figure 5


Figure 6

From Figure 5, Figure 6, we can see which part of the development process error is more, example 6, the algorithm errors and functional errors are the most so should be in the unit test or code review to pay attention to the quality of the two parts. In addition, we can also know where and how to correct the error code.

Application of ODC in functional testing

Here we take a look at the test, in the FVT (functional test), a major help fvt do a better job of the indicator is trigger, in the ODC trigger can be easily understood as what kind of test found this defect. In FVT we defined 4 trigger:coverage (the coverage here is not the meaning of test coverage in my general sense, it refers to the normal function, which is the functionality that any user will use, basic, simple functions), Variation (for some users who are familiar with the product, may be willing to use less commonly used creative methods or input to complete the same action or function, or simply to pick the wrong, in these attempts often found a lot of missing defect, such as ' boundary restrictions '), Sequencing (a task function is performed with a different operating process than before), Interaction (some errors can occur when two or more function modules interact with each other, such as the possibility of a system panic when starting some functions at the same time).

Let's take a look at the defect distribution in FVT by trigger, for example:

Figure 7

In Figure 7, we can see that this product in the general function coverage and change the sequence of operations sequencing test found more defect, this shows that the code also need to do more unit testing to reduce errors, so that we can understand the basic quality level of the product.

Figure 8

In Figure 8 we can see the coverage of the error is very few, the basic quality of the product can be guaranteed; variation a lot of errors (it seems that the Test team did a lot of testing in this area), sequencing and interaction errors are many, The two tests should be increased later in the test. Here we can clearly see what we've measured and what we need to do to increase the testing effort.

Figure 9

In Figure 9 we can see that there are many errors in variation, so the strength of the unit test can be a good solution to this problem, such as adding more boundary checks.

Application of ODC in system testing

Now let's take a look at SVT (System test), in system testing, the ODC defines another set of trigger, which are: Blocked (which defect to prevent the SVT from executing, most commonly fvt defect), Stress (The results of the stress test are most likely the data that the customer is most concerned about), Recovery (for data recovery and error handling), Restart (start and restart of the X system), Hardware configuration (hardware configurations), software Configuration (software configurations). Let's take a look at the defect distribution in SVT by trigger, for example:

Figure 10

In Figure 10 we see blocked defect too much, obviously this time to do a lot of SVT testing is unwise, then should let the product continue to perform functional testing until the blocked problem is reduced to acceptable.

Figure 11

In Figure 11, we can also see what tests have been done in SVT, where software configuration testing and stress testing are needed to be strengthened.

Figure 12

In Figure 12, we can see that the hardware configuration is defect more, so we should ask developers to pay more attention to this part of the code.

From the above analysis, we see that trigger in ODC tells us where we found the problem and what we should do.

Application of ODC for customer impact

So let's take a look at how ODC is impacting customers. What are the impact of software products on customers? The following aspects are defined in the ODC:

1.Installability
2.Security
3.Performance
4.Maintenance
5.Serviceability
6.Migration
7.Documentation
8.Usability
9.Standards
10.Reliability
11.Requirements
12.Capability

Figure 13

Here Figure 13 is a product of the defect impact distribution map, we can see that this product has a lot of problems in the "Capability performance", "Reliability Reliability", and "usability availability", then such products if sold to customers will be terrible, Then we should take the corresponding action to improve these aspects of the problem.

In ODC, other elements are defined to be used in combination to help us better understand where the problem is and help us make the right decisions.

When the tester discovers a problem and opens a defect, you need to define the following properties for defect:

1) Activity, which refers to the type of test found in the defect, such as UT, FVT, SVT and so on.

2) Trigger, the situation of the problem arises

3) Impact, affecting the customer's aspect

When the developer receives a defect and begins to modify the code, he needs to define the following attributes:

1) Target, where to correct the error, for example: design, code, etc.

2) Defect type, Defect, for example: algorithm, initialization, etc.

3) Qualifier: Only three, they are missing, incorrect and extraneous

4) Sources of Source:defect, such as: internal code, outsource code, etc.

5) Age:defect is the new code or the rewritten code

For details, refer to:

Figure 14

Back to top of page

Conclusion

In project and product development, we often think of the question: how effective are our tests? Are unit tests, functional tests, and system tests all done correctly? How do we reduce potential defect in demand, design, and development phases? Is our code complete and ready to move on to the next stage? Which of the defect have we found to belong to the previous version? How does the customer feel about the quality of our products? These are key issues that require us to improve our development and testing processes to make them more effective, further enhancing the quality of our products. So how to improve it? With ODC we can see which test method we are using to help us find more defect (basic functional tests, boundary condition tests or stress tests), and what kind of error is defect (whether it's an initialization problem or an interface problem, or something else), Whether the error is due to a missing code or a code error, defect is found in the old code or in the new code, and defect has a significant impact on the customer. By identifying the answers to these questions, we can improve the effectiveness of our development and testing, enhance the stability of our product modules, identify high-risk modules earlier, and finally make each version of the product better than the previous version.

Reprint: http://www.ibm.com/developerworks/cn/java/j-odc/

Introduction to ODC (orthogonal Defect classification)--orthogonal defect classification