[Reading Notes] Software Testing

Document directory

• 2.3.1 black box testing (functional testing)
• 2.3.2 design of the black box Testing Solution
• 2.3.3 white box test method (logical coverage method and structure test method)
• 2.3.4 Design of white box testing scheme
• 3.1.1 unit test content
• 3.1.2 unit test process
• 3.2.1 top-down
• 3.2.2 bottom-up

0. Reference Materials

Software engineering principles and applications: Chen shihong, Zhu Fuxi, Huang shusong, edited by Chen Lei; Wuhan University Press

1. Introduction: 1.1 What is software testing?

Software testing mainly checks and tests the software products, and finds faults and logical errors in the program in a timely manner to ensure the reliability of the software products. Software testing is a key step to ensure software quality and an important means to improve software reliability. Therefore, software testing is an important part of software engineering.

Software TestingContentThere are two aspects: documents and programs.

Software TestingPurposeIt is to improve the quality of software products with the least cost and the best effect.

Dijkstra has a brilliant remark on Software Testing"Testing can only prove that the program is wrong, and it cannot be guaranteed that the program is not wrong ".

 

1.2 software testing objectives

G. Myers elaborated on the purpose or definition of testing in his excellent book "software testing skills:

1. Program testing is the process of executing programs to discover errors;
2. A good test scheme is very likely to find the errors that have not been found so far;
3. A successful test is an error that has not been found before.

It can be seen that the traditional concept "program testing is to prove that there are no errors in the program", "successful testing is a test that fails to detect errors" is incorrect.

1.3 software testing principles

1. Pre-determine test results
2. Software developers (or departments) should not test their own programs
3. Develop strict test plans to prevent testing randomness
4. Designing and selecting test schemes should be conducive to error discovery
5. Focus on testing error-prone program segments
   • In many programs, errors appear in groups. These errors are often concentrated in a program. The probability of an error is compared with the error found in this program. This is the so-calledCluster symptom.
6. Save the test plan, test plan, error data statistics and classification, and the final analysis report.

2. Software Testing Methods

There are usually three methods for software testing. The first is program correctness proof, that is, verification; the second is static testing, that is, errors in the first program without executing the tested program; the third type is dynamic testing.

2.1 Program correctness proof

The Program Correctness proves the correctness of the program theoretically, and the logic of the program is proved to be correct.

In many cases, a complete form of proof may be unnecessary. In some cases, if you cannot perform a full test, it is impossible to implement a full proof of form. However, we often use the procedural correctness to prove the developed reasoning style to guide the testing process to enhance trust in the program, sometimes program proofs of some nature can be combined with tests of other nature.

2.2 Static Test

Static testing refers to finding out the program has errors without executing the program. This method analyzes and tests programs in manual and non-formal ways, and does not rely on computer testing. Practices show that static tests can detect about 30%-70% logical design errors and coding errors.

2.2.1 Function check (self-test)

Function check is also called self-test. The programmer compares the module functions (descriptions, algorithms, and syntax rules), flowcharts, and codes to read them repeatedly, and checks the syntax and logic errors of the program.

2.2.2 Group Check

Group inspection is a process of dynamic analysis of programs after the reporter reports the function description, flowchart, and self-testing of program code.

2.2.3 manual run check

Manual run check is performed by a human-played computer to execute the program and run the test scheme again according to the logic structure of the program, so as to identify the program errors for the tester to analyze.

2.3 Dynamic Testing

Static testing mainly checks the logic design and coding errors of the program, but it still has limitations in theory and practice, so it is still compared to dynamic testing.

The so-called dynamic test is to regard the program as a function, take every element in the function definition field as the input, run the program, and check whether the execution result of the Program falls within the function value field, to check the correctness, reliability and effectiveness of the program.

1. If the functions to be implemented by the product are known, you can test whether each function requirement is met. This is called "black box testing"
2. If you know the internal logic structure and processing process of the product, you can complete the internal operation test according to the specifications, which is called "white box test"

2.3.1 black box testing (functional testing)

The black box test method is also called the function test method.Software InterfaceBased on the analysis of the software functions, the test scheme is composed of representative elements in the definition field of the function. When using this method, the tester regards the program as a black box without considering the internal structure and processing process of the program. The purpose is to prove the effectiveness of functional requirements unrelated to the internal work of the program and seldom consider the internal logical structure of the software.

2.3.2 design of the black box testing scheme 2.3.2.1 division of equivalence classes

1. Rules for dividing equivalence classes
   • Value Range
   • Number of Values
   • Value Constraints
   • Value rules (for example, the file name must start with a letter)
   • Further division of equivalence classes
2. Test Plan

After the equivalence class is divided, a test plan should be designed based on the equivalence class. The main steps are as follows:

1.  
   1. Specify a unique number for each equivalence class.
   2. Contains reasonable equivalence classes. When designing a new test scheme, we should select as many test schemes as possible to cover the excluded reasonable equivalence classes.
   3. IncludeOneUnreasonable equivalence class. Design a new test scheme so that it only contains an unreasonable equivalence class that has not yet been included. Otherwise, the test results will be affected when multiple unreasonable equivalence classes are combined.

2.3.2.2 Boundary Value Analysis

A large number of errors in the program often occur at the boundary position of the input domain, rather than "intermediate ".The boundary value is relative to the input and output equivalence classes.

When using the boundary value analysis method to design a test scheme, the top boundary condition should be removed first, and the data that is just equal to, slightly less than, and slightly greater than the equivalence class boundary should be selected as the test data. The following are for your reference:

1. If the input condition specifies the value range, first select the boundary test condition for this range, and then write some examples of test conditions that just exceed this range.
2. If the input condition specifies the number of input data, it should be the minimum number, the maximum number, one less than the minimum number, one more than the maximum number design test scheme.
3. In addition to the input equivalence class, you should also consider the output equivalence class to select a test scheme.
4. If the input and output of a program are an ordered set (such as a linear table), focus on the first and last elements of the set.

The test scheme designed with the boundary value analysis method is more comprehensive and representative, and the probability of error discovery is higher. However, it is very difficult to find the boundary conditions and requires practical experience and creative work.

The main differences between the Boundary Value Analysis Method and the equivalence class division method are as follows:

1. The Boundary Value Analysis method selects one or several elements instead of any one of the equivalence classes. Each Boundary Value in the equivalence class is a test object.
2. The Boundary Value Analysis method not only considers the input conditions, but also uses the output equivalence class as the test scheme.

2.3.2.3 Error Prediction Method

The so-called error prediction method is to speculate some possible error types and special situations in the program through people's experience or intuition, and check these errors according to the test plan. The error inference method is mainly based on the experience accumulated by the tester in practice, so there is no definite test step.

2.3.3 white box test method (logical coverage method and structure test method)

The white box test method is also called the logic coverage method or the structure test method. It is based on the analysis of the internal logic structure of the software.Procedural detailsCheck carefully. The tester considered the program as a transparent box and checked the internal logical structure and processing process using the control structure of the process design. Using the white box test, the test solution determines the degree of test completeness for the Chengdu covered by the program logic.

2.3.4 Design of white box testing scheme

The white-box testing scheme can be used everywhere:

1. Make sure that all independent paths of the module are tested at least once.
2. Both true and false logic judgments are tested.
3. Execute all the Loops within the boundaries of the loop and the running boundary.
4. Test internal data structures to ensure their effectiveness

The following describes some common white-box testing solutions:

1. Statement Overwrite: Select sufficient test data so that each executable statement in the program can be executed at least once.
2. Decision coverage (branch coverage): A strong logic coverage. Sufficient test data is designed during testing, so that each of the true and false branches in the program can be executed at least once.
3. Conditional coverage: Use enough test data to make all possible results of each condition in the determination appear at least once.
4. Decision-condition override: Meets both the criteria for determining coverage and the criteria for conditional coverage.
5. Condition combination coverage: Select enough test data so that all possible combinations of conditions in each judgment expression can be executed at least once.

Note:

1. InDeterminationExpressions often contain multipleCondition.
2. Condition coverage does not necessarily include the decision coverage, and the decision coverage does not necessarily include the condition coverage.
3. Test data that meets the condition combination coverage must meet the criteria for determining coverage, condition coverage, and Decision-condition coverage.

3. steps and strategies for Software Testing

1. Unit Test
2. Joint Test
3. Validity test
4. System Test
5. Acceptance Test

Unit Test 3.1

Unit testing refers to the logical testing of a tested program as a single subroutine. The main task of unit testing is to verify whether its functions and interfaces indicate non-conformities and encoding errors.

Unit Testing is concentrated in each separate program block to eliminate fast logic, functional defects and errors, ensure that each block can be correctly executed as a unit and prepare for the previous test.

After static testing, you should focus on testing every unit in the program one by one, instead of testing the program as a whole for the following reasons:

1. Troubleshooting is easy.
2. This is conducive to joint testing.
3. Use unit tests to confirm detailed instructions.
4. Can be tested in parallel.

In unit tests, white-box testing is mainly used to test the functions as thoroughly as possible. black-box testing can help identify and respond to any reasonable and unreasonable input.

3.1.1 unit test content

1. Unit Interface. Before performing a unit test, you should first test the data stream of the Unit Interface, because the interface test is the basis of other tests. If the data stream cannot enter or exit correctly, other tests cannot be performed.
2. Test the local data structure. Local data structures should be carefully tested because local data structures are often the sources of errors.
3. Important Path test. As testing is not exhaustive, it is necessary to select an appropriate test plan to fully test the critical execution path in the unit. The objective is to find computing errors, comparison errors, or control flow errors.
4. Handle errors. Error Handling should be a part of the program's functions. In case of crude errors, execute the corresponding error handling path or terminate the program running.
5. Border test. Border testing is the last and most important task in unit testing.

3.1.2 unit test process

A unit is generally not an independent and complete program. Before performing a unit test, you must write a program for each unit test.DriverAndSupport Program.

In common applications,DriverIt is no different from the main program. It receives test data, transmits the data to the tested unit, and prints the relevant results.

Support ProgramInstead of the unit called by the Unit to be tested, the Support Program is sometimes "virtual subroutine", which provides the Unit Interface replaced by it, with as few data processing as possible, print and check the entry information, and return the control to the unit that calls it.

Drivers and support programs must be written separately, which requires a lot of effort and time. Because the design should be as simple as possible to reduce the overhead of software development. However, many units do not have the "simple" low overhead to act as unit tests. In this case, the testing work will be postponed to Joint Testing. Of course, a small number of drivers and support programs are still required.

3.2 Joint Test

On the basis of unit testing, each unit should be gradually connected according to the design requirements for joint testing, the main purpose is to find interface-related errors.

There are two methods for unit Joint Integration testing,Non-incremental testing and incremental Testing.

Non-incremental TestingIt is to test each unit separately, and then connect all units into a complete program system according to the design requirements.

Incremental TestingA unit to be tested is connected to a tested unit for testing. After the test is completed, the Unit to be tested is connected for testing. Only one unit is added at a time. In fact, this method completes both unit testing and joint testing.

 

The main advantages and disadvantages of the two methods are as follows:

1. The incremental testing workload is small. Non-incremental testing requires writing drivers and support programs for each unit, which requires a great deal of work. In addition, incremental testing can use tested units as part of the driver and Support Program, with a low workload.
2. Early error detected in incremental testing. The incremental test can detect the Unit Interface errors earlier, but the non-incremental test can only detect the interface errors between units after each unit is connected together.
3. The thoroughness of incremental testing. The incremental test connects tested units with the heartbleed component for testing. In the new environment, the tested units must be tested. As the number of units increases, some units need to undergo multiple tests. Obviously, this method is more thorough in program testing.
4. Incremental testing requires a lot of machine time.
5. Non-incremental testing can be performed in parallel.

In general,Incremental testing is a more suitable test solution..

 

The addition of additional test units isTop-down and bottom-upTwo unit join methods.

3.2.1 top-down

A top-down test is an incremental method of integrating the software structure. Starting from the main unit (main program), it moves down the control layer of the software, and then gradually connects each unit, there are two strategies for connecting tested units and new units: depth first and breadth first.

3.2.1.1 basic principles for determining the test sequence

1. Test key units as early as possible.
2. Test the unit that contains input and output operations as soon as possible. Because the unit with these functions is tested, it facilitates the input of test data and the output of test results.

3.2.1.2 Test Procedure

1. The main control unit is used as the test driver, and its subordinate units are replaced by the Support Program.
2. Based on the selected connection Policy (depth-first or breadth-first), replace the subordinate support units with the actual units one by one.
3. One unit is tested for every connected unit.
4. Regression tests can be performed to partially or fully repeat the tested units to ensure continuous discovery of new errors.
5. Repeat steps 1-4 until the entire software system is completed.

3.2.1.3 advantages

1. Early detection and correction of main interface errors.
2. No driver is required.
3. Users can see the main functions of the system in the early morning.
4. The duration of testing machines can maintain a certain degree of stability throughout software development.

3.2.1.4 disadvantages

1. It is required to write the Support Program, which is complex and difficult to design the test scheme;
2. Difficult to establish test conditions and determine test results;
3. Easy to forget necessary tests;
4. Parallel testing is difficult.

3.2.2 bottom-up

The bottom-up test is the opposite of the top-down test. It is based on the bottom-up development technology and generally starts the test after the program design is complete, start the connection and test from the bottom-layer unit of the software structure.

3.2.2.1 Test Procedure

The bottom-up test is usually carried out at the later stage of system development, that is, after system analysis, system design, and program design are completed, the test starts. At this time, all the units of the system already exist. The implementation steps are as follows:

1. Combine underlying units to implement a specific software sub-function;
2. Compile a Driver (Test Control Program) to coordinate the input and output of the test;
3. Test Unit Set
4. Remove the driver along the software structure and connect the new unit.

3.2.2.2 advantages

1. No support program is required;
2. Parallel tests can be conducted as early as possible;
3. Identify key unit errors as soon as possible;
4. It is easy to establish test conditions;
5. It is easy to judge the test results.

3.2.2.3 disadvantages

1. Write the driver;
2. Integration is required as a subsystem first;
3. Interface error found late;
4. The system profile is formed late.

3.3 Effectiveness Test

The software is effective if its functions and performance can meet users' reasonable requirements.

Black box testing is generally used for software effectiveness testing.

The effectiveness test needs to verify whether the software meets the requirements of the user. In addition, it is necessary to verify whether the documents are complete and accurate, the portability, compatibility and maintainability of the software.

An important part of the effectiveness test process is to review the software configuration to ensure that all components of all software configurations (such as user files, design file code tables, and test files) complete, its quality meets the requirements, supports the necessary details in the maintenance phase, and also needs to arrange the directory.

3.4 System Test

System testing is a software used for effectiveness testing as part of the entire computer system. It is combined with hardware, peripherals and other systems to conduct a series of joint tests and effectiveness tests on computer systems.

The purpose of the system test is to compare the software with the system requirement definition and find out where the software and definition do not conform or conflict with each other.

The following types of system tests are available:

1. Function test. Check whether the program meets "what to do" according to the specifications ".
2. Capacity Test. The purpose is to let programs process a large amount of data and determine whether these programs can process a specified number of tasks.
3. Strength test. Strength Testing refers to whether the program can complete the specified maximum working capacity (number) within the unit time, so that the program can work under high load conditions.
4. Convenience test.Whether the interface structure and output results are incomprehensible, whether the error information is concise and understandable, and whether the operation is convenient.
5. Security Testing. Carefully design some test schemes to check program security.
6. Test the storage capacity. The test content includes memory and external storage capacity, temporary file or file overflow and size.
7. Configuration test. It mainly tests the resources required by the program, such as the requirements for testing input/output devices, operating systems, and languages.
8. Compatibility/conversion performance test. Especially for those new versions of the program, both to test the compatibility of the new version and the old version, but also to consider the future version expansion issues.
9. Installability test. It is very important to test the installation process of installation steps, especially system software.
10. Reliability Test.The Software Reliability refers to the probability that the software runs correctly in a specific environment within a given period of time.
11. Resume the test. When a hardware fault causes a program or data error, test whether the entire system can resume normal operation.
12. Maintainability test.Tests the maintenance means provided by the system, such as diagnostic programs, dump programs in memory, and maintenance procedures.
13. Document testing. Check whether the documents provided to users are complete and accurate.
14. Procedure Test. Test whether the procedures to be followed by various personnel (such as system operators and database management personnel) are reasonable and feasible.

3.5 Acceptance Test

The acceptance test is carried out with the user's participation. It mainly uses actual data for testing. The purpose is to verify that the system can meet the user's needs. The test content is basically the same as that of the joint test, errors found during the test are often in the System Requirement Specification.

 

 

2010.9.21:))

-End-

4. Glossary

1. Regression testingIn the software maintenance phase, a test is performed after the software is modified to check whether the software is modified correctly.
2. The so-calledSoftware ReliabilityIndicates the probability that the software runs correctly in a specific environment within a given period of time.
3. Briefly describeWaterfall Model.
   Problem definition> feasibility analysis> requirement analysis> outline design> detailed design> code> test> Maintenance
4. Load TestingIt generally refers to the practice of simulating multiple users to use the system services at the same time to imitate the intended use of a software program.