Architecture Design of Automatic Software Testing

Chapter 2 Preface
There are already a lot of automated testing tools available on the market, but it is difficult to find or pay for the testing tools that meet your needs, for software products running on the Linux/Unix background, you can develop an automatic testing tool that not only meets the software testing requirements, but also saves a lot of money.
The design of this automatic testing system architecture is a software product based on Linux/Unix background. The architecture concept is derived from the practical experience of mainstream testing tools and predecessors.
Automatic software testing is actually an idea. Whether it is a mainstream testing tool on the market or a self-developed testing tool, it is only a tool. The key is how to organize a tool, how to apply the tool to software testing.
　　Chapter 3 system architecture
2.1 Design Philosophy
1. composition of Automatic Testing: Automatic Testing consists of the following parts: (1) Automated Testing Tool (2), test case (3), simulated interface (4) tested software (5) automatic compilation and Installation
2. automatic Testing Tool: the main control program of the automatic testing tool does not need to understand the business, and all the business logic and test data are written in the test case, the master program executes instructions in each test case to complete the test in each test case. The test results are recorded in the specified file.
3. test Case: business logic and test data are reflected in the test case, A complete test case should include test condition presets, test steps, input and output of each test step, expected results, acquisition of actual test results, comparison of actual test results with expected test results. Each test action is an operation instruction. Each operation instruction includes the instruction ID, input, and output. The active test tool performs the automatic test by executing the commands in the test case.
4. black box testing: the tested software is a black box for the automated testing tool. The automated testing tool does not care about the internal logic and business flow of the tested software, interfaces of the software to be tested and the input and output of each interface. The Automatic Test Tool inputs test data to the tested software and obtains the test result from the corresponding interface. If the test result is consistent with the expected result, the test passes. Otherwise, the test fails.
5. Simulated interfaces: interfaces that deal with the tested software must be simulated. When necessary, you can set the returned values of the simulated interfaces for the tested software to achieve the purpose of testing.
6. automatic compilation and testing: automatic compilation is to automatically compile the software on the source code management server and upload the software to the test server. Automatic Testing is to automatically update or install the tested software, start the automatic testing tool to execute the automatic test cases, record the test results, and send the test results to relevant personnel by email.
7. test environment recovery: after a test case is executed successfully or fails, the test environment after the test case is specialized must be restored.
8. Reuse Policy: common modules are extracted and called by other modules or functions to improve the availability of modules and reduce program redundant code.
9. Composition of the automatic test tool module:
(1) Master Program: Read test cases and operation instructions for each test case, call different command interfaces according to different operation commands to execute the commands for each test case, and record the test results.
(2) command interface: it serves as a bridge between the test case operation instructions and the implementation of specific test steps, and corresponds to the test case operation instructions, the command interface is what is required to implement the operation commands of the test case. After the action is completed, the Operation Result of the action is returned to the master program.
(3) DRIVER: the driver is a program that implements specific command operations. It is called by the command interface to complete the specific test and return the test result to the command interface.
(4) auxiliary functions: Implement auxiliary functions of the system, such as case file processing, underlying socket processing, file processing or database operations.
2.2 System Architecture
2.3 system flowchart
　　Chapter 4 Automatic Test core design
3.1 system composition
The automatic testing tool consists of the Main Control Program, command interface, driver and auxiliary functions.
3.1.1 master program
The control module does not understand the business. It writes all the business logic and test data in the case. The control module reads all the test cases to be tested and the actions of each operation item in each test case, call different interface processing modules to implement the entire test process.
Process of the control module:
1. Read the command to obtain the command to be executed and the configuration file.
2. Analyze the command and command parameters. For example, if the command is to view help, print the help prompt. If the command is to execute the test case, start to execute the test case of automatic testing.
3. Analyze the configuration file to obtain the directory where the test case is stored, the ID file of the case to be executed, the database connection information, and the parameters used in the test case. (Note: In the execution of the test case, a full test case may not be executed, and some actions in the test case may not need to be executed. You can configure the actions in the configuration file or test case)
4. Store the test case directory and Test ID file to obtain the test case to be tested.
5. Obtain the action commands and input data to be executed in each test case.
6. Execute test cases one by one, and call different interfaces to execute different actions according to the Action commands in the test case.
7. After a test case is completed, obtain the test result of the test case and output the test result to the result record file.
8. continue to the next case until all test cases are tested.
3.1.2 command INTERFACE
It is a bridge between the test case operation instructions and the implementation of specific test steps. In contrast to the test case operation instructions, the command interface is what needs to be done to implement the test case operation instructions, after the action is completed, return the operation result of the action to the master program. The interface module processes data with interfaces, including input and output, and data processing results. After the master program reads the operation commands in the test case, it calls the command interface corresponding to these operation commands to execute the corresponding operations. If the operation is successful, the master program continues the next operation, otherwise, the test is terminated, and the test case passes only when all the commands of the test procedure are successfully executed.
To facilitate management and calling, all interfaces are instantiated in an interface factory. After corresponding command interfaces are implemented, these command interfaces must be added to the interface factory, the interface factory calls different interface instances based on the interface type in the case.
The command interface is actually what each command in the test case requires, such as inputting test data in the test case, executing the corresponding test steps, obtaining the test results, and comparing the test results, after the command interface completes a command, it returns the operation result of the command to the master program.
For example, for the ticket check command interface, what the command interface needs to do is to initialize a single file for listening, get the ticket file, and compare the obtained ticket file with the expected ticket, return the comparison result to the master program.
Another example is the SMS receiving and sending command interface. The command interface is used to send the MT message, collect the status report, and compare the status value of the received status report with the expected status value, return the comparison result to the caller. After receiving the local Mo, compare the received local Mo with the expected Mo and return the comparison result to the caller.
3.1.3 driver
A driver is a program that implements specific command operations. It is called by the command interface to complete the specific test and return the test result to the command interface. The driver is the completion of a specific operation.
For example, the ticket check driver is used to track a single file in a specific ticket format and parse the tracing result into a ticket, the ticket check command interface calls the ticket driver to compare the child segments of each ticket. After the driver completes the check, the result is returned to the instruction interface.
3.1.4 auxiliary functions
Implement auxiliary functions of the system, such as case file processing, underlying socket processing, file processing or database operations. [Page]
3.2 Test Cases
3.2.1 composition of Test Cases
Test Cases include business logic, test steps, and input and output. Using XML to store test cases can improve the readability, versatility, and maintainability of test cases. However, one bad thing is that the test cases may look huge and lightweight, but clear, even if they are not familiar with the test cases, as long as you read the test, you can quickly get a general understanding of the business process. In addition, the business logic is reflected in the test cases, which also enhances the scalability of the system, to add some test steps or modify test data, you only need to modify the test case.
The test case consists of the test case id, test case description, and test operation instructions. The test case ID is the case ID of the test system. The test description includes the brief description of the test case, the name of the test product, the test function, the test version, the test type, and the author of the test case, test Operation commands are the core of the test case. Each test step is an operation command. The operation order is related to the business logic or test step. Each operation command includes:
(1) operation command ID, which is unique in test case;
(2) The name of the operation instruction interface, the instruction interface name provided by the automatic test tool, can appear multiple times in a test case;
(3) The action name of the operation command interface. The specific operation name of the command interface provided by the automatic test tool can appear multiple times in a test case, the operation command interface name and operation command interface action name of the Automatic Testing Tool call the functions of different command interfaces to complete related actions;
(4) Description of operation instructions;
(5) Information about the simulated interface. If you need to use the simulated interface, you need to input the information about the simulated interface in the test case. The information about the simulated interface should be parameterized;
(6) input data, that is, the test data required to perform this operation, the expected results, or the delay wait time.
(7) output data, that is, the output result of this operation step.
A test case is a complete test process consisting of different operation steps. Each operation step is an operation instruction. A complete test case may include the operation instructions:
(1) preset test environments, such as modifying some special configurations, setting the return values of analog interfaces, and checking some data;
(2) test procedure: a test case may have multiple test steps. Each test step is an operation instruction. The input of the operation instruction is the data to be input for the test, action is the test operation to be executed, and the test case is completed by a series of operation commands;
(3) Check the test results. In a test case, multiple checkpoints may exist. Each checkpoint is an operation command. The input of this operation command is the expected test result, the action of the Operation Command is to obtain the test result and compare the test result with the expected test result. If the test result is equal, the test passes; otherwise, the test fails;
(4) restore the test environment. After the test is completed, the raw data or related resources of the test environment should be restored.
To improve the scalability and flexibility of the test, the test data in the test case should be parameterized, so that even if the test data changes, you only need to modify the parameter value, you do not have to modify some data of all test cases one by one. These parameters and parameter values can be stored in the configuration file and referenced by the test case. The automatic test tool reads the value of this parameter from the corresponding configuration file based on the parameter name of the test case.