Automated Test Architecture Design



The 1th Chapter Preface

Now there are many automated testing tools on the market, but the testing tools to meet their needs are difficult to find or difficult to pay for their expensive costs, for the software products running in the Linux/unix background, self-developed an automated test tool, not only to meet the requirements of software testing, but also save a large amount of money.

The design of this automatic test system architecture is based on the Linux/unix background running software products, architecture ideas, from the mainstream testing tools and predecessors of practical experience.

Software automated testing, in fact, is a kind of idea, whether it is the mainstream testing tools in the market, or self-developed testing tools, are just tools, the key is how to organize a tool, how to apply the tool to software testing.

2nd Chapter System Architecture

2.1 Design Ideas

1. The composition of the automatic test: The automatic test mainly consists of several parts: (1), Automatic test tool (2), test case (3), Analog interface (4) automatically compiled and installed by the test software (5)

2. Automatic test Tool: The main control program of the automatic test tool does not need to understand the business, all the business logic and test data are written in the test case, the master executes the instruction in each test case to complete the test of each test case, and the test result is recorded in the specified file.

3. Test cases: Business logic and test data are reflected in test cases, a complete test case should include test condition presets, test steps, input and output of each test step, expected results, acquisition of actual test results, comparison of actual test results with expected test results. Each test action acts as an operation instruction, each operation instruction includes the instruction ID, the input and the output, the active test tool is completes the automatic test through these instructions which executes the test case.

4. Black box test: The tested software is a black box for automated testing tools, and the automated test tool does not care about the internal logic and business processes of the software being tested, the interfaces of the tested software concerned, and the inputs and outputs of each interface. The Automated test tool enters test data into the tested software, obtains the test results on the appropriate interface, and passes the test if the test results match the expected results, otherwise the test fails.

5. Analog interface: interface with the tested software, need to simulate, if necessary, can set the analog interface to the test software return value, to achieve the purpose of testing.

6. Automatic compilation and Testing: Automatic compilation is automatically compiled to the source control server software, the software uploaded to the test server. Automated testing is the automatic updating or installation of the tested software, initiating automated test tools to perform automated test cases, documenting test results and sending the test results to the relevant person by mail.

7. Recovery of the test environment: After a test case is completed, whether the test succeeds or fails, you need to restore the test environment that was specialized in the test case.

8. Reuse policy: The common module is extracted and called by other modules or functions to improve the utility of the module and reduce the program redundancy code.

9. The Automatic test tool module consists of:

(1) Main control program: Read the test case and the operation instruction of each test case, according to different operation instructions, call different instruction interface to execute the instruction of each test case, record the test result.

(2) Instruction interface: is the test case operation instruction and the concrete test step realization Bridge between, and the test case's Operation instruction corresponds, the instruction interface is realizes the test case the operation instruction to do the thing, the action completes the action result to return to the main control program.

(3) Driver: The driver is the program that implements the specific instruction operation, the instruction interface calls to complete the specific test work, and returns the test result to the instruction interface.

(4) Auxiliary function: to realize the auxiliary functions of the system, such as case file processing, the underlying socket processing, file processing or database operation.

2.2 System Architecture

2.3 System Flowchart

3rd Chapter Automatic Test Core Design

3.1 System composition

The automatic test tool consists of a master program, a command interface, a driver, and an auxiliary function.

3.1.1 Master Control Program

The control module does not understand the business, the business logic and test data are written in the case, the control module reads all the test cases that need to be tested and the actions of each action item in each test case, invoking different interface processing modules to implement the entire test process.

Control Module Processing Flow:

1. Read the command to get the command to execute and the configuration file.

2. Analyze the commands and parameters of the command, for example, if the command is to view Help, print a Help prompt, or if the command is to execute a test case, start the test case for the automated test.

3. Analyze the configuration file, get the test Case storage directory, the ID file of the case to be executed, the database connection information, and the parameters used in the test case. (Note: Because the entire test case may not be completed during the execution of the test case, some actions in the test case might not need to be performed and can be configured in a configuration file or test case)

4. Obtain a test case that needs to be tested based on the test case storage directory and the ID file of the test test.

5. Get the action instructions and input data that each test case needs to execute.

6. Execute each test case one by one, invoking different interfaces to perform different actions based on the action instruction of the test case.

7. After a test case test is complete, obtain the test results for the test case and output the test results to the result record file.

8. Continue with the next case until all of the test cases have been tested.

3.1.2 Instruction Interface

It is the bridge between the test case operation instruction and the concrete test step realization, and the instruction interface is the necessary thing to implement the operation instruction of the test case, and the operation result of the action is returned to the main control program after the action is completed. The interface module is responsible for data processing with the interface, including input and output, and must include the results of the data processing. After reading the operation instruction of the test case, the main control program invokes the instruction interface corresponding to the operation instructions, and if the interface performs the operation successfully, the active program continues the next operation, otherwise the test terminates, and only the instructions of all the test steps are successful, the test case passes.

To facilitate management and invocation, all interfaces are instantiated in an interface factory, and after implementing the corresponding instruction interface, these instruction interfaces need to be added to the interface factory, and the interface factory invokes different interface instances based on the type of interface in the case.

Instruction interface, in fact, is the implementation of each instruction in the test case required to do things, such as input test case test data, perform the corresponding test steps, obtain test results, compare the test results, the instruction interface to complete a command after the operation of the command to return the results to the main control program.

For example, the command interface, instruction interface needs to do is to initialize to listen to the single file, to obtain the Word file, the acquisition of a single file with the expected statement to compare, and return the comparison results to the main control program.

Another example: SMS Receive Send command interface, the work of the instruction interface is to achieve send MT, receive status report and the status report received state value and the expected state value to compare, return the comparison results to the caller; local Mo is received, and the local Mo received is compared to the desired MO, and the results are returned to the caller.

3.1.3 Drivers

A driver is a program that implements a specific instruction operation, is called by the instruction interface to complete a specific test, and returns the test result to the instruction interface. The driver is the person who completes the operation.

For example, the single-word check driver, for the specific word format of the single-file tracking, and the ability to parse the trace results into a single-word, the single-check instruction interface invokes the single-word driver to complete the specific individual sub-segments of the comparison, the driver completes the check to return the results to the instruction interface.

3.1.4 Auxiliary function

To implement the auxiliary functions of the system, such as case file processing, the underlying socket processing, file processing or database operation.

3.2 Test Cases

Composition of the 3.2.1 test case

Test cases include business logic, test steps, and input and output, using XML storage test cases to improve readability, versatility, and maintainability of test cases. But there is a downside to this is that the test case will look large, not light enough, but very clear, even if the test case is completely unfamiliar to the person, as long as it is natural to see a general understanding of the business process, and the business logic reflected in the test case, but also enhance the scalability of the system, If you need to add some test steps or modify the test data, you only need to modify the test case.

The test case consists mainly of the test case ID, the test case description, and the test action instruction. The test case ID is the case ID of the test system, the test description includes a brief description of the test case, the test product name, the test function point, the test version, the test type, and the author of the test case, the test action instruction is the core of the test case, each test step is an operation instruction, The order of operation instructions is related to the business logic or test steps. Each operation instruction includes:

(1) Operation instruction ID, unique in test case;

(2) Operation instruction interface name, the instruction interface name provided by the Automatic test tool, can appear multiple times in a test case;

(3) Operation instruction interface action name, the name of the specific operation of the instruction interface provided by the Automatic test tool, can appear multiple times in a test case, the automatic test tool operation instruction interface name and operation instruction interface action name call different instruction interface function to complete the related action;

(4) Description of the operation instruction;

(5) The information of the analog interface, if need to use to the analog interface, need to the analog interface related information re-test case input, analog interface related information should be parameterized;

(6) Input data, that is, perform this operation step (7) output data, that is, the output of this procedure.

A test case is a complete test process consisting of a different sequence of operations, each of which is an action instruction, and a complete test case may include these operations directives:

(1) Preset test environment, such as modify some special configuration, set the return value of analog interface, check some data, etc.

(2) test procedures, a test case may have multiple test steps, each test step is an operation instruction, the input of the operation instruction is to test the data to be entered, the action is required to perform the test operation, test case test is a series of operational instructions to complete;

(3) Check the test results, there may be multiple checkpoints in a test case, each checkpoint is an operation instruction, the input of the operation instruction is the expected test results, the action of the operation instruction is to obtain the test results and compare the test results with the expected test results, if the same test pass, otherwise the test fails;

(4) Restore the test environment, the test environment should be restored after the original data or related resources.

To improve the scalability and flexibility of testing, test data in test cases should be parameterized so that even if the test data changes, only the values of the parameters need to be modified, rather than having to modify some of the data for all the test cases individually. The values of these parameters and parameters can be stored in the configuration file, referenced by the test case, and the Automated test tool reads the value of this parameter according to the name of the parameter of the test case to the corresponding configuration file.

Storage of 3.2.3 Test cases

For different product test cases, should be stored in a different home directory, for the same product test cases, according to different functions in different sub-directories. Each home directory or subdirectory should have a test case ID file, and the test case in this file will be automatically tested by the automated test tool. In the test case ID file, each behavior is a test case, and each test case consists of a serial number, a test case ID, and a description of the test case. Where the test case is stored, the automated test tool can be obtained from the configuration file.

3.3 Automated Test execution

The writing and testing of 3.3.1 test cases

After the automated testing tools are implemented, much of the work that follows is the writing and testing of test cases, transforming test cases in test case systems into automated test case scripts based on the specifications of business logic and automated test cases, and testing these scripts after automated test case scripting. Ensure that the automated test case script is correctly executed and correctly tested for the functionality described in the test case. Before testing with automated testing tools, it is important to test whether automated test tools and cases are properly tested for related functions, otherwise the results of automated tests are not credible and automatic testing is meaningless.

3.3.2 Automated Test Execution

After the automated test case is written, the automated test can be tested without intervention.

(1) The ID of the test case that requires automated testing is written in a file, and the automated test tool executes only the test cases in this file;

(2) The contents of the automatic test case, the database connection, the IP and port of the analog interface are written in the configuration file, the automatic test tool will read the test case to the directory specified in the configuration file, and will also read the database connection information and the simulation interface information used by the automatic test tool;

(3) The parameters used in the automatic test case are written in the parameter configuration file, and the Automatic test tool reads the value of the parameter in the parameter configuration file according to the parameter of the automatic test case instead of the parameters in the automatic test case;

(4) Specify the output file of the test results, the automatic test tool after testing a test case, the test results of this testing case output to the test results file, the test results file each line represents a test case, each test case output includes the test case ID, The functional description of the test case and the results of the test case;

(5) Automatic test tool in the test case, the need to record the test log, including the test case ID, read the contents of the test case, test steps, test results of each test step, comparison of test results;

(6) Automated test tools to automatically perform the required test cases, and record the test results, test engineers to review the test results after the completion of the test, test successful test case means that the functional test pass, for test failure test cases, need to analyze the reasons for the log, If the environment or test case or the automated test tool is caused by a test environment or test script, you will need to log the bug and notify the developer of the problem found in the test case where the test failed.

4th chapter Automatic compilation and Automatic test

4.1.1 Automatic Compilation

Automatic compilation is automatically compiled on the source control server, the results of the compilation are analyzed, and the files that are compiled successfully and are required by the automated test environment are updated to the test environment. The required input test data or expected results or latency wait time.

The process of automatic compilation can be divided into:

(1) Update the source code on the server to compile all the relevant code, need to compile the path of the code in the configuration file read;

(2) Automatically compile the source code needed to compile;

(3) Analysis of the source code compilation results, only the successful compilation of the executable file for automatic testing only makes sense, the compilation results uploaded to the test server and sent to the appropriate personnel;

(4) package and upload the files that need to be updated to the test server and send the results to the relevant people.

Automatic compilation, you can write a script to the crontab to call the automatic compilation program, to achieve unattended compilation automation.

4.1.2 Automatic Test

The automated test described here is to automatically update or install the software being tested, automatically start the tested software, then run automated test cases for automated testing, and send the results of automated tests to the relevant test or developer.

The process of automated testing can be divided into:

(1) to automatically compile the results of the upload directory to obtain automatic compilation results, analysis of automatic compilation results, if the automatic compilation fails the automatic test end, send test results to the relevant personnel, or a step;

(2) to the FTP test software installation file upload directory to obtain the FTP upload results, analysis of FTP upload results, if the FTP failed to automatically test the end, send test results to the relevant personnel, or proceed to the next step;

(3) to the test software installation file upload directory to obtain the installation files;

(4) Stop the original running test software, if it is a newly installed test environment, do not need to perform this step;

(5) Software Installation or update: If it is a new test environment for the installation of software, if the existing test environment is updated by the test software;

(6) Start the tested software;

(7) Call Automatic test tool to test automatically, record test result;

(8) After all the automated test cases are tested, analyze the test results and email the test results to the relevant test and developer.

Automatic compilation, you can write a script to the crontab to call the automatic test program, to achieve unattended test automation.

The configuration file should include the following information:

(1) The path and file name of the result file and FTP result file automatically compiled;

(2) The path of the installation file or update file of the software being tested;

(3) The path and file name of the automatic test result;

(4) The path and file name of the script to stop the original test software;

(5) The path and file name of the script to start the software being tested;

(6) The path and file name of the script to start the automatic test software;

(7) e-mail address of the test result sent.