Application of Genetic Algorithm in black box testing

In software testing, black box testing is mainly for functional testing of modules. The most common method is to divide the input of software into several equivalence classes based on the functional specification of the software, and run the software multiple times to check whether the software can meet the requirements of different equivalence classes. However, in practical applications, some modules are too large or there are too many input parameters. The testing work required after the equivalence classes are divided may be a huge task. In this case, selecting the best test cases becomes an important task for testers.
Genetic algorithms are an optimal method to imitate the biological genetic and Evolutionary mechanisms. They take some behavior similar to genetic genes, such as cross-reorganization, variation, selection, and elimination are introduced into the improvement process of algorithm solving. One of the characteristics of a genetic algorithm is that it retains a number of local optimal solutions at the same time, and seeks a better solution through cross-reorganization or solution variation. Compared with greedy algorithms, genetic algorithms are more likely to find the global optimal solution, while greedy algorithms are prone to local optimization and fail to reach the global optimal.
If the genetic algorithm can be effectively used in the black box test to help the tester select the best test case, it will bring great help to the test work.
1. Application Method
Before designing a specific algorithm, we first introduce the basic algorithm of the genetic algorithm. The algorithm framework is as follows:
Step 1: Initialize the solution. Select the P semi solution as the initial solution and use the best solution as the tentative (optimal) solution.
Step 2: solution improvement: if the termination condition is met, the tentative solution is output and the algorithm is terminated. Otherwise, perform the following operations:
(1) Cross-reorganization of the solution: Two or more solutions are selected from the P solution for cross-reorganization to obtain a new solution and repeat the operation several times.
(2) solution variation: some variations are randomly added to the solution to generate a new solution.
(3) Local Search: The local search method is used to improve the new solution. If a better solution is obtained than the explain solution, update the explain solution.
(4) Select the P solution as the next generation's ultimate solution based on certain criteria, and update the tentative solution.
Go to step 2.
After learning about the algorithm framework of the genetic algorithm, what we need to do further is how to transform different equivalence classes into the hidden solution of the genetic algorithm in the software black box test, and how to set the standard of the solution, how to set appropriate conditions for termination.
We assume that the input parameters of a software module include five: A, B, C, D, and E. After a reasonable equivalence class division, each parameter has five different equivalence classes: a1 ~ A5,..., E1 ~ E5. We adopt a generalized concept of genetic algorithm regression solution. General Genetic Algorithms usually define the form of regression solution as binary data strings, such as 111010 and 010001, when the input of different equivalence classes is used as the scheme solution, we set the scheme solution form as (based on the above assumption): a3b1c2d4e5, a2b2c4d1e3, and so on. In this way, we have solved the problem of termination. In terms of the merits and demerits of the solution and the setting of termination conditions, we need to use tools as the standard.
The purpose of software testing is to improve the reliability of the software. The termination condition is of course that the software has achieved the purpose and requirements of testing. The solution standard is the opposite of the software quality, that is, the higher the probability of software failure, the better the test case (an input solution. In the document 2, a test-Based Failure Data Model jbfdm (Jade Bird failure data model) is proposed in conjunction with Peking University's qingbird black box testing environment ). With this model, we can achieve 2:
(1) provide consistent failure data modeling, collection and management of reliability measurement processes to support reliability measurement;
(2) Evaluate the effectiveness of the test plan, operation diagram, and test method by using the data collected from testing and software on-site.
The purpose of software testing is to discover errors. In the black box testing, errors are represented by software failures. However, software errors are not a sufficient condition for software failure. In other words, not all errors are exposed during testing or running, so the purpose of the black box test is to try to run the test case to make the software invalid and find errors. When evaluating test cases, we use the following data to indicate the advantages and disadvantages of the test cases:
A = P + λ/m + μ × F
Where a indicates the adaptability adaptation in the genetic algorithm, P indicates the probability of the test case occurring in the actual probability, M indicates the average failure time (MTTF), and F indicates the failure level. Because tests are intended for use, it is difficult to understand the high degree of adaptability of high probability test cases; and M-the longer the average failure time, the test cases should not be easy to find software errors, therefore, the lower A, the less F indicates that the software is seriously invalid in some special circumstances (such as causing crashes or damage to the instrument). At this time, the test case and its future generations must be highlighted, so the larger the value of. λ and μ are coefficients corresponding to specific tested software modules. In practical applications, because the possibility of software failure is not very high, the genetic result is often a lot of future generations of test cases with high probability. Therefore, we should design an accurate generation algorithm based on the probability of occurrence for the tested software. The specific algorithm framework 1 is shown in.
The algorithm is described as follows:
* 1. Each input parameter usually has a probability (which can be defined in advance). You can simply add the values to obtain the probability of the test case. However, when the input parameters have a strong correlation, this method cannot accurately calculate the probability of a test case. One solution is to set the Coupling Degree of the input parameters. The probability of simultaneous crossover and mutation of a genetic algorithm is directly proportional to the coupling degree. That is, for input parameters with high Coupling Degree, the probability of simultaneous crossover and mutation is high, and vice versa.
* 2. Set a counter to check whether the test meets the requirements. Each time a new test case is run, the test counter is incremented by one. When the first failure or failure is detected, the counter is added with two. If the generated genetic descendant invalidates the software, the counter is increased to 22. Likewise, when the test cases generated by the genetic algorithm make the software invalid for N consecutive times, the counter is added to 2N. At the same time, record all the testing conditions (this work is completed by the peripheral testing environment, such as the qingbird black box testing environment of Peking University ). If a serious error occurs, terminate the test and check the program. If the genetic descendant of the consecutive k-generation test case runs well, the counter value is increased by 2 K. The value of K is determined by the number of equivalence classes and the number of input parameters of the software to be tested. When the value of the test counter reaches the value of all the black box test case equivalence classes (for the example above, the value is 55 = 3125), the test is completed. When the generation of grandchildren, child generations, and parent generations are completely the same, the algorithm must end, because no new results will be produced in the test. Therefore, we need to set an end condition. This condition is stronger than the counter condition.
* 3. Multiple test cases can be generated for each set of test cases, and the test cases left behind are determined based on the fitness function size to form a new test case group.
From the algorithm diagram and description above, we can see that if a test case causes a problem in the running of the software (that is, a software error occurs), its future generations will also suffer from the software error, the algorithm quickly discovers these best test cases and provides results. Testers can give them to developers to solve these problems. If the software itself is indeed of excellent quality, these test cases and their different generations cannot find failure, and the algorithm can also end as soon as possible, rather than completing all test cases (although theoretically, we want the test to run all test cases as much as possible ).
2 Results
The algorithm in the previous section has no obvious advantages over running all test cases. Especially for testing, the algorithm does not accelerate the running of test cases, as if it has reduced the running speed. In fact, the algorithm itself is not used to accelerate the running of test cases, its purpose is to find a set of best test cases. In fact, it is almost impossible for many modules to run all test cases or even all equivalence classes.
The example in the preceding section shows that the input equivalence class is roughly 55 = 3125. If a module has 10 inputs and each input has 10 equivalence classes, the input equivalence class is 1010. It takes one minute to run an equivalence class (many cyclic operation modules may take more than one minute). It takes several months to run all the equivalence classes. At this time, the advantages of genetic algorithms are shown.
To sum up, this paper proposes a principle of Using Genetic Algorithms to find the best test cases. It can complete the black box test of the software module in a short period of time and provide the test results and good test cases. With this algorithm principle, you can set or modify the test integration environment in the test integration environment, which greatly improves the test efficiency.

From: http://testing.csdn.net/page/c312dfcd-40b3-433f-9bae-6fc20b338755