Due to time and cost constraints, the most critical issues in software testing are:
Which subset of all possible test cases is most likely to find the most common errors?
Test method:
Black box testing
- Equivalence partitioning)
1. strictly control the increase of test cases and reduce the number of other test cases that must be designed to achieve certain set goals of "rational test"
2. It covers most other possible test cases.
After dividing the equivalence class, we can say that if no error is found in the test of an element in the set, therefore, tests on other elements in the set are unlikely to detect errors.
There are two steps to design a test case using the equivalence classification method: (1) determining the equivalence class; (2) generating a test case
(1) determine the equivalence class
External Condition, valid equivalence class (representing valid input to the Program), invalid equivalence class (any other possible input condition-that is, incorrect input value)
Determining equivalence classes is a heuristic process. Some guiding principles are as follows:
- If the input condition specifies a value range (eg. "quantity can be from 1 to 99"), then a valid equivalence class (1 <quantity <99) and two invalid equivalence classes (quantity <1, quantity> 99) should be determined)
- If the input condition specifies the number of values (eg. "one to six car owners can be registered"), a valid equivalence class and two invalid equivalence classes should be determined (no car owner or six car owners)
- If the input condition specifies a set of input values, and the program considers that each value is processed differently (eg. "The vehicle type must be a bus, truck, taxi, train or motorcycle"), then a valid equivalence class and an invalid equivalence class should be determined for each input value (eg. trailer)
- If the input condition specifies "must be" (eg. "the first character of the identifier must be a letter"), then a valid equivalence class (the first character is a letter) and an invalid equivalence class (the first character is not a letter) should be determined)
(2) Generate Test Cases
The process of generating test cases using equivalence classes:
- Set a different number for each equivalence class
- Write new test cases to cover as many effective equivalence classes as possible that are not covered, and know that all valid equivalence classes are covered by test cases.
- Write a new use case to overwrite one and only one unoverwritten invalid equivalence class until all invalid equivalence classes are covered by the test case: overwrite the invalid equivalence class with a single test case, this is because some specific input error checks may block or replace other input error checks.
The so-called Boundary Condition refers to the states in the input and output equivalence classes that happen to be in the boundary, exceed the boundary, or below the boundary.
Process used to generate test cases:
-
- Splits the specification into executable segments.
- Determine the causal relationship in the Specification Description. A reason refers to a clear equivalence class of input conditions or input conditions. The result is an output condition or System Conversion.
- Analyze the semantic content of the specification and convert it into a Boolean graph that connects the causal relationship. This is the so-called cause
- Add the annotation symbol to the graph, indicating the reason and result that cannot be linked due to syntax or environment restrictions.
- By carefully tracking the status changes in the graph, the causal indicator is converted into a finite-item discriminant table. Each column in the table represents a test case
- Convert columns in the decision table into test cases
The basic symbols in the causal indicator:
Suppose that the value of each node is 0 or 1 or 0, indicating "not exist", and 1 indicating "exist. The identity function indicates that if A is equal to 1, B is also 1; otherwise, B is 0. the not function indicates that if A is equal to 1, B is 0; otherwise, B is 1. the or function indicates that if A, B, or C is equal to 1, D is 1; otherwise, D is 0. The and function indicates that if both A and B are equal to 1, C is 1; otherwise, C is 0.
White-box testing)
- Statement Overwrite
- Measure whether each executable statement in the tested code is executed.
int foo(int x, int y) { return x / y; }
Test Case, x = 10, y = 5.
- The tester's test results will tell you that the code coverage rate has reached 100% and all test cases have passed. Unfortunately, we did not find the simplest bug. For example, if we set Y to 0, a division by zero exception will be thrown.
- Overwrite Determination
- This criterion requires that sufficient test cases be written so that each judgment has at least one output result of "true" and "false", and each statement is executed at least once. In other words, each judgment must have "yes" and "no" results, and each entry point must be called at least once.
- Conditional coverage
- In case of condition coverage, you need to write enough test cases to ensure that all possible results of each condition in a judgment are executed at least once, and each entry point must be called at least once.
- Decision/condition override
- Execute all possible results of each condition in a judgment at least once, and execute all possible results of each condition in each judgment at least once, execute all possible results of each judgment at least once, and call each entry point at least once.
- Multi-condition coverage
- This criterion requires that a sufficient number of test cases be written to combine all possible condition results in each decision and to execute all entry points at least once.
- The reason is that the results of certain conditions in the expressions "and" and "or" may block or hinder the interpretation of other conditions. For example, if a condition in the "and" expression is "false", you do not need to calculate the subsequent conditions in the expression.
public void foo(int a, int b, int x) { if (a > 1 && b == 0) { x = x / a; } if (a == 2 || x > 1) { x = x + 1; } }
- Statement overwrite: Ace-A = 2, B = 0, x = 3
- Decision coverage: ACD, Abe-A = 3, B = 0, x = 3; A = 2, B = 1, x = 1
- Conditional coverage: Abe-A = 2, B = 0, x = 3; A = 1, B = 1, x = 1
- Multi-condition coverage: The test case must cover the combination of 8
- A> 1, B = 0
- A> 1, B <> 0
- A <= 1, B = 0
- A <= 1, B <> 0
- A = 2, x> 1
- A = 2, x <= 1
- A <> 2, x> 1
- A <> 2, x <= 1
- A = 3, B = 0, x = 4
- A = 2, B = 1, x = 1
- A = 1, B = 0, x = 2
- A = 1, B = 1, x = 1
Test policy:
1. If the Specification Description contains a combination of input conditions, use the causal analysis method first.
2. Use the Boundary Value Analysis Method in all circumstances. Analysis of input and output boundaries
3. Valid and invalid equivalence classes should be determined for the input and output, and the test cases confirmed above should be supplemented if necessary
4. Add more Test Cases Using Error Prediction Technology
5. Check the logic structure of the program for the test case set above. Criteria for determining coverage, condition coverage, criteria for determining/condition coverage, or multi-condition coverage should be used.
Art of testing: Design of Test Cases