Software Testing Economics _ Software Testing

Given the appropriate definition of software testing, the next step is to determine whether the software test can detect "all" errors. We will prove that the answer is in the negative, even for a very small program. In general, it is impractical and often impossible to discover all the errors in a program. This basic question, in turn, hints at the economics of software testing, the expectations of testers for software being tested, and the way in which test cases are designed.

In order to meet the challenge of test economics, some strategies should be established before testing begins. Black-Box testing and white-box testing are two Yun common strategies that we'll discuss in the next two sections.

Black box test

Black-Box testing is an important test strategy, also known as data-driven testing or input/output-driven testing. When using this test method, the program is treated as a black box. The test objective is completely unrelated to the internal mechanism and structure of the program, but focuses on the environmental conditions in which the Discovery program does not run correctly according to its specifications.

In this approach, the test data comes entirely from the software specification (in other words, you don't need to understand the internal structure of the program). If you want to use this method to discover all the errors of a program, the criterion is "exhaustive input test", and all possible input conditions are used as test cases. Why did you do that? For example, in the Triangle test program, tried three equilateral test cases. This does not ensure that all equilateral are correctly judged. The program may contain a special check for the 3842,3842,3842 of the side length and indicate that the triangle is an irregular triangle. Since the program is a black box, the only way to determine the existence of this statement is to experiment with all the input.

To be exhaustive. Test this triangle program, you may need to create test cases for all valid triangles, as long as the triangle side length is within the bounds of the Yun large integer values allowed in the development language. These test cases are astronomical in themselves, but this is by no means exhaustive, when the program points out that -3,4,5 is an irregular triangle or 2,a, 2 is a isosceles, the problem is exposed, in order to ensure that all such errors can be found, not only with all valid inputs, It also has to be tested with all possible inputs. Therefore, in order to raise the test triangle program, you actually need to create an unlimited test case: This is of course impossible.

If it's so hard to test this triangle program, it's even more difficult to test a larger program, imagine, if you want to do a black box exhaustive test on a C + + compiler, not only to create a test case that represents all of the valid C + + programs (actually, this is an infinite number), You also need to create test cases (infinity) that represent all invalid C + + programs to ensure that the compiler can detect that they are not valid. That is, the compiler must test to make sure that it does not perform actions that should not be performed-such as successfully compiling a syntactically incorrect program.

This problem can become particularly serious if your program uses data storage, such as an operating system or database application. For example, in a database application such as a flight reservation system, transactions such as database queries, flight bookings, and so on, need to be determined with the execution of the last transaction, so that not only all valid and invalid transactions are tested, but all possible transaction sequences are tested.

The above discussion shows that the exhaustive input test is not achievable, there are two meanings, one is that we can not test a program to ensure that it is error-free, and the second is the software testing of a fundamental issue to be considered is the economics of software testing. In other words, because the exhaustive test is not possible, the goal of the test cast is to yun the number of problems discovered by the limited test cases to obtain Yun good test results. In addition to other factors, to achieve this goal, it is also necessary to have a glimpse of the interior of the software and to make reasonable but not unassailable assumptions about the procedure (for example, if the triangle procedure treats 2,2,2 as equilateral, then there is reason to think that the procedure will make the same judgment for 3,3,3).

White box test

Another test strategy, called a white-box test or a logic-driven test, allows us to examine the internal structure of the program. This test strategy tests the logical structure of the program and obtains the test data (unfortunately, the Code of the program is often overlooked).

Our goal here is to set up a test method that is similar to the exhaustive input test in black box testing for the strategy of reaching a test. There may be a solution that will be executed at least once for each statement in the program. But it is not hard to prove that it is far from enough. This method is often referred to as the exhaustive path test, which will be further explored in the future, and is not described here. The so-called exhaustive path test, that is, if a test case is used to execute all possible control flow paths in the program, then the program may be fully tested.

However, there are two problems with this assertion. First of all, the number of different logical paths in the program Yun may reach astronomical numbers. The applet shown in Figure 2-1 shows this. The diagram is a control flow diagram in which each node or circle represents an sequentially executed statement segment, usually ending with a branch statement. Each edge or arc represents a transformation of the control (branch) between the statement segments. Figure 2-1 depicts a program with a 10~20 line statement that contains an iteration of the Do Loop 20 times. In the Do loop body, contains a series of nested IF statements. Determining the number of different logical paths is also equivalent to determining the number of all different paths from point a~ to point B (assuming that all of the judgment statements in the program are independent of each other). This number is about 1014, or 100 trillion, from 520+519 ... 61 calculates, 5 is the number of paths within the loop. Since most people find it difficult to have an intuitive concept of this number, imagine that if you could write, execute, and validate a test case in every five minutes, it would take about 1 billion years to test all the paths. If you can be 300 times times faster, complete a test every second, it will take a long 3.2 million years to complete the work.

......

This article is reproduced from the 51Testing Software test Network, view the full text: http://www.51testing.com/html/78/n-817978.html