Test generation based on software testing

1. Introduction

Designing test inputs and corresponding expected outputs is one of the most basic technical activities of any testing organization. Both the test input data and the corresponding expected output are written to the test case. A collection of test cases is a test set. Currently, a number of guidelines, techniques, and support tools exist to generate test cases. The next step is to introduce a lot of guidance and techniques based on test generation.

2. Software Requirements

Software requirements are the basic starting point of design testing. In the initial stages of software development, software requirements are only in one or several people's brains. Strict requirements can be obtained by using modeling elements such as UML use case diagrams, sequence diagrams, and state diagrams. Further, the use of formal requirements specification language such as Z, S, RSML, the strict requirements of the software can be further converted into formal requirements. Although the complete formalized requirements specification is a useful document, it is often used to obtain software requirements by using appropriate modeling mechanisms. UML, as an efficient modeling language, integrates many different modeling elements into a unified framework, which defines the software requirements strictly and formally by these modeling elements.

Three types of representations of requirements specification:

(1) Informal norms

(2) Strict specifications

(3) Formal specification

Of course, it can sometimes be a mixture of these three forms. The higher the formalization of the requirements specification, the more opportunities are available for automated testing.

A) The software input domain can be extracted from the informal and strict requirements specification, and the input domain is the basis of the test design. The informal and rigorous requirements specification includes the following testing techniques:

(1) Equivalence class division

(2) Boundary value analysis

(3) Causality diagram analysis

(4) Decision table

These techniques can be used to select a relatively small number of valid test cases from the usual large software input domain as test sets.

b) Formal requirements mainly include the following test techniques:

(1) Model-based technology

(2) Specification-based technology

(a) and (b) The software testing techniques introduced are in the black box testing category. And some of these techniques can be enhanced when there is software source code.

3. Test Case Selection Issues

Set D as the input field of software p, test case selection problem refers to: Select a subset of the test case T, the elements in T as input to execute p, during the execution will find defects in P. In general, there is no algorithm for constructing such a set of test cases, but it is possible to effectively expose some specific types of defects by running heuristics and the set of test cases generated based on the model method. Therefore, the key of the test case selection problem is how to construct the test case set T, which is a subset of D, so that the use of set T can find out the defects in software p as much as possible. The main reason why the test case selection problem is difficult is that the input domain of software p is large in size and high in complexity.

A software input domain is a collection of all legitimate inputs that the software may receive during execution. The legal input set of the software is determined by the software requirements. In many practical problems, the size of the software input domain is very large, may contain many elements, but also complex, and these elements may have multiple types, such as integers, strings, real numbers, booleans, and structs.

In general, the size of the software input domain is very large, which makes it impossible for testers to use all possible input values to carry out exhaustive testing of the software under test. The so-called exhaustive test means that testers use all the elements in the input domain to test the software on a case-by-case basis. The complexity of the input fields increases the difficulty of selecting test cases.

In some cases, because of the constraints between input and timing, it is difficult to describe the software input domain, which is unavoidable in the software testing process. Therefore, a variety of test case selection methods came into being, testers use these methods to select a small subset from the software input domain, in order to achieve the purpose of testing software. These test case selection methods are described below, along with the advantages and disadvantages of each approach.

4. Equivalence class partitioning (often one of the most commonly used test design techniques)

When the method of equivalence class partitioning is used to test the design, it is required that the testers divide the input domain into as few subdomains as possible, and in the process of dividing the sub-domain, according to strict mathematical definition, each subdomain 22 is required to intersect each other. Each subdomain is called an equivalence class.

The principle of equivalence class partitioning: The software is tested with any input from the same equivalence class, and the software can output the same result. Under such preconditions, the tester simply chooses an input from each of the divided equivalence classes as a test case, and the test case of N (n equivalence Class) constitutes a complete set of test cases for the software.

Of course, the equivalent class partition of the same input field, the result may not be unique. Therefore, the set of test cases produced by the equivalence class partitioning method may also be different. Even if the equivalent classes are divided by two testers, they may choose different sets of test cases. The rate of failure detection for this set of test cases depends on the test design experience of the tester, familiarity with the software requirements, availability of the software source code, and familiarity with the source.

4.1) Defect Location

A collection of all the inputs of a software can be divided into at least two subsets: one contains all normal and valid inputs, denoted by e, and the other contains all exceptions and illegal inputs, expressed in U. These two collections can be further divided into subsets, so that the software runs differently for different subsets. The equivalence class partitioning method is to select the appropriate input from either set or its sub-set as the test case in order to discover flaws in the software that cause it to run abnormally.

For example, suppose software A is entered as an integer representing the age of the person. Assuming that the legal value of the age should be within the [1,120] range, therefore, the input set can be divided into normal input set E, its value range is [1,120], the exception input set U, its value range is except [1,120] all the remaining integers. Further, it is assumed that the software processes all inputs between [1,61] according to the requirements R1, and processes all inputs between [62,120] according to R2. Therefore, based on the expected behavior of the software, the set E can be further divided into two subsets. Similarly, for an illegal collection of u, the software handles all inputs less than 1 in one way, and all inputs greater than 120 are processed in another way, and you are divided into two subsets. In this way, software A has two input fields containing the normal input and two containing the input of the exception, and the equivalence class partitioning method is to find the defect of software A by selecting the test case from the 4 input regions.

4.2) Relationship and equivalence class division

In set theory, a relationship refers to a set of N-tuples. For example, the method Addlist calculates and returns the sum of a list of integers, so addlist defines a two-tuple relationship. Any of the two tuples in the relationship consists of a list of integers and the sum of the integers of the list, such as ((1,5), 6), (( -3,14,3), 14), ((), 0), and so on. The corresponding relationship of addlist can be defined in the following form:

Addlist:l->z

Where L is a collection that contains a list of all integers, and z is an integer set. As we can see from the previous example, each software, program, or method defines a relationship. In fact, this conclusion is correct as long as the definition fields (that is, the input collection) and the range (that is, the output collection) are defined correctly.

For example, suppose there is a flaw in the method addlist that the method fails when the input integer list is empty. In this case, the method Addlist defines the previous relationship l->z, even in accordance with the requirements specification, and it does not run normally. The relationship it actually defines is as follows:

Addlist:l->z∪{error}

When dividing the input domain of software, testers often take the following relationship:

R:i->i

Where I is the input field and r is the relationship on I, R defines an equivalence class, which is a subset of I.

4.3) equivalence classes of variables

(1) Range of values:

The range of values can be defined in both a display and an implicit way. For example, the range of speed speeds can be defined as [60,90], while the range of area areas is implicitly defined. For speed, testers can determine input values outside the range of values, while for area, although it is possible to determine the input value outside the range of values, but because the software and hardware system run by the test data representation limits, it is possible that the tester can not enter values outside these values range.

Division principle: (↓ illegal input equivalence class, ↑ delegate of the legal input equivalence class)

Equivalence class: An equivalence class within a range of values; two equivalent classes outside the range of values

Constraint: speed∈[60,90] equivalent class representative: {{50}↓,{75}↑,{92}↓}

area:float;area>=0 {{ -1.0}↓,{15.52}↑}

Age:int;0 <= Age <= 120 { -1}↓,{56}↑,{132}↓}

Letter:char {{j}↑,{3}↓}

(2) String:

The partitioning of a string will use its semantic information for the period

Partitioning principle: (∑ represents an empty string)

Equivalence classes: At least one class that contains all the valid strings and a class that contains all the illegal strings. Legitimacy is determined by the length of the string and other semantic characteristics

Constraint: fname:string (name, non-empty string with a length of not more than 10, and can only have letters) equivalent class is represented: {{∑}↓,{sue}↑,{sue2}↓,{too Long a name}↓}

vname:string (variable name, non-empty) {{∑}↓,{shape}↑,{address1}↑,{long variable}↓}

(3) enumeration:

If the software behaves differently for different values of the variable, then each value itself constitutes an equivalence class, as is the case with the Boolean variable. For enumeration types, such as for certain ranges of values, it is possible that an illegal test input value could not be determined. For example, the Boolean input variable up accepts only true and false values, and all of its possible equivalence classes will contain only legal values.

Equivalence class Partitioning principle:

Equivalence class: Each value corresponds to an equivalence class

Constraint: Auto_color∈{red,blue,green} equivalent class representative: {{red}↑,{blue}↑,{green}↑}

Up:boolean {{true}↑,{false}↑}

(4) Array:

An array is a set of elements of the same type, and the length of the array and its types can be used as the basis for equivalence class partitioning.

Equivalence class Partitioning principle:

Equivalence class: An equivalence class containing all valid arrays, an equivalence class for an empty array, and an equivalence class containing all arrays greater than the expected length

Constraint: Jave array:int[] equivalent class delegate: {{[]}↓,{[-20,10]}↑,{[-9,0,12,15]}↓}

AName = new Int[3]

(5) Composite data type:

The input data that contains two or more independent properties for each of the two. For example, arrays in Java, as well as records or structs in C + +, are composite types. This input type is used when testing a component module of the software, such as a function or an object. When you divide the input of this composite data type into equivalent classes, you need to consider the legal and illegal values of each attribute of the input data.

4.4) Unified points and diversified points

One of the input domain partitioning methods is to consider only one input variable at a time, so that each input variable forms a division of the input field, which we call a unary equivalence class division, referred to as unified. In this case, there is a relationship R for the input field of each variable, the input field of the program is based on R, and the number of variables, the number of partitions, each of which contains two or more than two equivalent classes.

Another method of input domain partitioning is to treat the Cartesian product of all input variables as the input field I of a program and define the relationship R on I. This method produces only one partition, and the division consists of several equivalence classes. We call this division method a multivariate equivalence class division, referred to as the plural division.

The test cases obtained by using the multivariate partitioning method are often more fully tested than the test cases obtained by using the unified method. On the other hand, the number of equivalence classes produced by the diversification method will increase exponentially with the number of input variables.

4.5) The complete process of equivalence class partitioning

(1) Determining the input domain

(2) Equivalence class division

(3) Combinatorial equivalence class

(4) Determine the non-measurable equivalence class: Some input data combinations are not generated during the actual testing process, and the equivalence class containing such data is the non-measurable equivalence class. There are many reasons for generating non-measurable equivalence classes. For example, suppose the GUI of a software tests it, that is, the data can only be entered through the GUI. The GUI interface contains only all valid inputs and does not allow invalid input. There are some constraints in the software requirements, which cause some equivalence classes to be non-measurable

4.6) GUI design and equivalence class

The specific implementation of the GUI must be considered when designing the test case. In some cases, the design of the test is governed by the GUI design. For example, the GUI design process may require the GUI to provide only the legal value of the input variable as much as possible. Of course, the GUI needs to be tested separately for these requirements.

5. Boundary value Analysis

Boundary value Analysis (BEM) is an effective test case selection method, which can be used to find software defects at equivalence class boundaries. The equivalence class partitioning method chooses the test case from the equivalence class, and the boundary value analysis method chooses the test case from the equivalence class boundary or the boundary. Of course, the test cases generated using these two methods may overlap.

In the process of designing test cases, both boundary value analysis and equivalence class partitioning are used. In addition to using equivalence class partitioning to determine boundary values, you can also use relationships between input variables to determine boundaries. Once the input domain is determined, the main steps to generate the test case using boundary value analysis are as follows:

(1) Divide the input fields using the unary partitioning method. At this point, the number of input variables will be divided into how many. If multivariate partitioning method is used, it can only form a division of input domain.

(2) Define boundaries for each division. You can also use specific relationships between input variables to determine boundaries.

(3) Design test cases to ensure that each boundary value appears in at least one test input data.

Attention:

(1) To determine the input domain boundary value, we need to carefully consider the relationship between the input variables, to obtain those in the input, output variable equivalence class is not obvious boundary.

(2) using the Cartesian product of a single variable equivalence class to divide the input domain, more test cases will be obtained.

6. Classification method

The classification method is a systematic method to generate test cases from software requirements. The essence of the classification method is that the tester transforms the software requirement into the corresponding test specification, in which the test specification is composed of the corresponding software input variables and the various categories of the Environment object. Each category is divided into several options that correspond to one or more values for the software input variable and the state of the Environment object. The constraints of each option are also included in the test specification to ensure that a reasonable and effective test set is generated. By inputting the written test specification into the test framework generator, the test framework can be obtained and the corresponding test script can be generated according to the test framework. A test framework is a collection of options, one of which corresponds to a category. A test framework can also be thought of as a template for one or more test cases, combining these test cases to form one or more test scripts.

This method includes both manual and automated steps. Here are the detailed steps to use this method:

(1) Analysis function specification

In this step, the tester determines all the functional modules that can be independently tested. For large software systems, functional modules may correspond to individual subsystems that can be independently tested, while subsystems can be further subdivided into sub-modules that can be independently tested. According to the specific test object to determine the time of termination of this decomposition process.

(2) Identify categories

The functional specifications of each module are analyzed, the corresponding inputs are determined, and the objects in the environment (such as files) are determined. Next, determine the characteristics of each parameter and environment object. The so-called feature is a category, some of which are clearly defined, while other implicit features require careful analysis of functional specifications to obtain.

(3) Classification

The tester will test the function module according to the different situations in each category. One case is an option, and each category will contain one or more cases. Each category can be divided into at least two subsets, one containing all the correct values, and the other containing all the wrong values.

For software deployed in a networked environment, you need to consider various network failures, and carefully consider other situations, such as database invalidation. At the same time, testers need to take full account of the software under test in the actual use of all possible to encounter the valid and invalid situation.

(4) Determining the constraint conditions

A test of a functional module, including a combination of the parameters of the module and all the options of the Environment object. Because some constraints must be met between input parameters, it is not possible to have a subset of the option combinations. In any case, the constraints between the options will be determined in this step. In step 6, the test Case Builder will generate only valid test frameworks based on these constraint relationships.

Constraint relationships can be represented as attribute lists or selection expressions.

The list of attributes is as follows:

[Property p1,p2 ....]

Where property is a keyword, p1,p2, and so on, represents the name of each attribute. You can assign an attribute to each option.

The selection expression is a connection to those defined properties in the attribute list, in the form of the following:

[If P1]

[If P1 and P2 and ....]

property lists and selection expressions are typically placed after each option. When the property list following an option is [ERROR], it means that the option is an error state, and if the property list is [single], the option cannot be combined with the options of other parameters or environment objects when the tester builds the test framework in step 6.

(5) write test code (heavy)

After assigning a list of attributes and selecting an expression for each option, testers can write complete test specifications. A Test specification language (Test Specification LANGUAGE,TSL) is required for the writing of testing specifications with strict syntax.

In step 7, the tester evaluates the test framework that was generated by step 6, and you can repeat step 5 multiple times if you are dissatisfied with the test framework or if there is a redundant framework found. The so-called redundancy framework is a framework that is composed of options that are impractical or not likely to occur in real-world use. In this case, the tester will rewrite the test specification and rebuild the test framework one at a time by performing step 6.

(6) Processing test specifications

The TSL specification written in step 5 will be processed by an automated test framework generator, generating several test frameworks. The tester analyzes the test framework to find out the redundant content in the framework that tests the tested software in the same way. In this case, testers can rewrite the test specification or simply reject the redundant content.

The test framework is not a test case. From the test framework it is easy to generate test cases that contain specific input values and expected outputs. It is important to note that the test framework also contains information about the environment object, which facilitates proper configuration of the test run environment before the test is executed.

The test framework is a combination of the various options based on the constraint relationship, and the option to be marked as error or single can only generate one test case, not a combination of other options.

(7) Evaluate the output of the generator (if you need to modify, re-execute step 5)

In this step, the main task of the tester is to check whether the test frameworks that are generated in step 6 contain redundant use cases and if some use cases are missing, go to step 5 (rewrite the test specification), and re-execute step 6.

(8) Generate test script

The test cases that are generated from the test framework are combined into test scripts. The so-called test script is a set of test cases. Typically, the same set of test cases with the same environment settings can effectively improve the efficiency of test drives executing test cases.

The classification method is basically a systematic method of combining equivalence class partition and boundary value analysis.

When writing test specifications, the test team is required to study the software Requirements specification, software design instructions and other software documentation carefully and observe the tested software. For large software systems, the writing of test specifications can be assigned to each member of the Test team. While most of the key steps in the classification method need to be done manually, using tools that process the TSL method will effectively improve the efficiency of generating test cases and writing documents, as well as reducing errors in test cases.

7. Causality Diagram Analysis

Equivalence class partitioning and Boundary value Analysis Select test Cases, one of which is based on the unified of the input field, and the other is based on the multivariate partitioning of the input domain.

Causality diagrams, also known as dependency models, are primarily used to describe the dependencies between software input relationships (that is, "cause") and software output results (i.e. "results"). Cause and effect diagrams can visually represent various dependencies. Here, the causality diagram is a graphical representation of the logical relationship between the input and the output, which can also be expressed as a Boolean expression. Testers can select different combinations of inputs from the causality diagram as test cases. When generating test cases, the combination explosion of the number of test cases can be effectively solved by using a specific heuristic method.

"Cause" refers to any input conditions that can affect software output in software requirements. "Result" refers to the response of the software to the combination of certain input conditions. The "results" here can be an error message displayed on the screen, a pop-up window, or an update to the database. But the "result" is not always visible to the software user "output", in fact, it may be an internal test point in the software, in the test process by testing the test point to determine whether the intermediate results of the software operation is correct. For example, an internal test point might be at the entrance of a function to indicate that the function has been activated.

For example, the requirement "Distribute food only when the DF switch is on", contains a reason "df switch is on" and a result "distributing food". The requirement implies the relationship between the "DF switch is on" and the result "distributing food". Of course, other requirements may also require additional reasons to produce the results of "distributing food".

The general process of generating test cases using causality diagram methods:

(1) Carefully review the software Requirements specification, determine which are the causes, what are the results, and assign a unique identity to each cause and result. Note that some results are also the cause of other results.

(2) Describe the relationship between cause and result by causality diagram.

(3) The causality diagram is converted into a decision table of a finite entrance, and is referred to as the decision table.

(4) Generate test cases according to the decision table.

7.1) basic symbols in cause and effect diagrams

The following constraints often exist between the reason (that is, the input condition):

(1) Rejection constraint (E): Only one of the input conditions is true

(2) Containment constraint (I): At least one of the input conditions is true

(3) Requirement constraint (R): If one condition is true, another condition must be true

(4) Unique constraint (O): The input condition has only one true

Similarly, there is a binding relationship between the results. The causality diagram method provides a "shielding relationship (M)" between the results. That is, the same input criteria cannot have two results at the same time, such as online shopping, the results may be imported or not shipped (two results only one).

7.2) Create causality diagram

The process of causality diagram analysis consists of two main steps:

(1) The analysis of software requirements to determine which is the cause and which are the results. At the same time, the causal relationship between the cause and the result is clarified, as well as the constraint relationship between the cause and the result, which gives a unique identification for each cause and result, which is convenient to be referenced in causality diagram.

(2) construct a causal diagram to express these dependencies that are extracted from the software requirements. When the cause, the number of results is large (usually more than 100 reasons, or more than 45 results), the use of incremental method is more appropriate.

7.3) Generate decision table from causality diagram

Each column in the decision table represents a combination of an input value, which is a test case. Each condition and result has a corresponding row in the decision table. Therefore, the decision table can be thought of as a matrix of n * m, where n represents the sum of the number of conditions and results, and M represents the number of test cases.

The value of the element in the decision table is either 1 or 0, depending on whether the corresponding conditions are true or false. For a result, if the value is 1 or 0, it corresponds to the result "appears" or "does not appear" respectively. The following is an algorithm for generating a decision table (DT) from causality diagram (CEG), which is recorded as CEGDT.

Input: Contains reason c1,c2 .... CP and Results Ef1,ef2,....... The cause and effect diagram of EFQ ceg.

Output: A decision table containing n=p+q rows and m columns dt, where m relies on the relationship between causes and results in the causal diagram.

Begin of CEGDT

/* I is an index that represents the next result that will be processed.

Next_dt_col: The next empty column in the decision table.

Vk: A vector of length p+q consisting of 1 and 0.

Vj:1 <= J <= p, indicating the state of the reason CJ

Vl:p <= l <= p + Q, indicating whether the result of EF (L-P) has occurred

*/

Step 1: Initialize the DT to a null decision table.

Next_dt_col = 1;

Step 2:for i = 1 to Q, perform the following steps.

2.1 Select the next result that will be processed.

Let E = Ef1

2.2 All cause combinations that lead to the occurrence of the result e are asked.

Assume that the result e occurs. Starting with E, the causal diagram is traced back to determine the conditions that lead to the occurrence of E-c1,c2, ... A combination of CP. You can use the heuristic method below to avoid the combination explosion of the number of test cases. At the same time, ensure that the cause combination can meet the constraints between the reasons.

Set V1,V2,......,VMI as a cause combination that can cause the result e to occur because the E is assumed to occur, that is, E is in the 1 state, so at least one cause combination is true, that is, MI >= 1. Based on the result that EF (l-p) occurs for the cause combination in VK, place VK (L) = 1 or VK (l) = 0,p < L <= p + Q.

2.3 Construction Decision Table

Starting with Next_dt_col, add V1,V2,.......,VMI as a subsequent column of the decision table.

2.4 Fix the next available column in the decision table.

Let_next_dt_col = Next_dt_col + mi

At the end of the algorithm, NEXT_DT_COL-1 is the number of test cases generated.

End of CEGDT

7.4) Heuristic method to avoid combinatorial explosions

Backtracking of cause and effect graphs gives a combination of reasons that set an intermediate node or result to a 1 state or 0 state. This use of the "brute" approach will produce a very large number of reasons for the combination. In the worst case, if there are n related to a result e, then the cause combination of E 1 states can be up to 2^n.

When a test case is generated from a combination of reasons, a large n value will cause the number of test cases to be too large. This combination explosion can be avoided by using a simple heuristic method associated with the "∧" node, "or" (∨) node.

Of course, heuristic methods are built on the assumption that some types of errors occur less frequently than other types of errors. Thus, while generating test cases in a heuristic way is likely to significantly reduce the number of test cases generated, it also ignores some test cases that could have detected errors. Therefore, use the heuristic approach with extreme caution and use this method only if the number of test cases produced by other methods is large enough to be meaningless.

7.5) Generating test cases from the decision table

Each column of the decision table produces at least one test case. Note that when a condition in a causality diagram can be satisfied with more than one value, a combination can generate multiple test cases. For example, consider the following reasons:

C:x < 99

The above condition C can be satisfied by multiple values, such as x = 1,x = 49; Similarly, there are multiple values that do not satisfy the condition C, such as x = 100,x = 999. This allows testers to choose different values for input variables when generating test cases from columns in the decision table.

Although there are several options for determining the value of an input variable, as long as they satisfy the requirements in the decision table, it is recommended that the choice be made so that the newly generated test case is not the same as the test case generated by another method, such as boundary value analysis

8. Predicate-based test generation

　

　　

　　

Test generation based on software testing