Development of Test Tools

Release date: Author: hehe_debbie Source: 51testing software testing network editing

Font family: medium to large | previous article | print
| I want to contribute | recommended label: Test and development of software testing and testing tools

Every time you perform a test architect, you will be faced with the problem of testing tools. Especially in System
Verification Department, their case is to simulate the use of products by users. The features of the products are constantly added, but the user behavior can be followed and complicated, is a combination of various behaviors. Repeated tests seem to be a waste of time for testers. In this case, automated testing tools have become a magic weapon to improve testing efficiency and free testers.

There are many things to note about how to develop test tools.

I remember when I first went to the WiMAX department as a test architect, the WiMAX functional testing group needed a software group to help them intercept normal packets and generate error packets. When I spoke with the tester, the tester said the tool was not easy to use. I went to ask the software staff again. They were surprised. If the functions have been fully implemented as required and all functions have been done, how can we refuse to use them? I looked at their implementations and found that the basic functions are actually implemented, but they are actually useless. Testers not only need this software to intercept and modify packets, but also need to use it flexibly in the actual test process to implement the overall test. The software personnel did not perform tests and could not correctly understand the needs of the testers. In addition, the communication between the two departments was not smooth on the 15th and 16th floors, which made them unable to understand each other, you cannot do what you really need. I sat down with the software staff and carefully told them how I needed to use the tool, what interfaces they needed, and how testers could access and manage the software, which configurations need to be visualized, which configurations need to be called by automated tools, what outputs are required, what statuses are displayed in real time, what data requires statistics, and what logs need to be output to the unified log document, and so on. They suddenly realized that the interception and modification of packets was only a basic requirement and there were so many things to do.

Therefore, the ideal situation for developing test tools is to find someone who is both proficient in development and testing. However, such a person is hard to find. A more common approach is to allow software and testing personnel to work closely together and understand each other to truly complete usable products.

In my opinion, a good automated testing tool requires attention to the following features:

First, data, configurations, and functions must be separated.

Second, try to use an existing stable public library

Third, the advantages and disadvantages of using C, Java, TCL, and Python

Fourth, encapsulate basic functions and provide calling interfaces for Secondary Development of testers

Fifth, use a unified and editable format to record data, configurations, and logs.

Sixth, do not make great efforts for perfection, nor use overly complex and flexible testing frameworks. be specific and simple.

7. friendly graphic interface, in line with the usage habits of testers

8. Pay attention to system efficiency and adopt multi-process to take advantage of multi-core CPU

Ninth, the test result acquisition, statistics, and log analysis page is provided.

10. Consider how to accommodate existing test cases

We have used the mobile metrics test suite. It is said that only nine people in the company are developing and maintaining this product, but this test suite is powerful, it is better than what 100 people in big companies do. It is definitely a model for us to learn. This suite is written in C and Java, and the underlying call and message processing are both C, to improve system performance. The user interface is both Java and very friendly. It provides a large number of interface calls and a reasonable encapsulation granularity. Testers can use a very simple Java language to implement very complex test behaviors, such as inbound, outbound, and handover. Standard Java calls are provided to allow testers to easily use other standard functions, such as generating random numbers, conditional statements, cyclic statements, network connections, file read/write, and packet analysis.

There is also the spirent automated test suite, which is user-friendly on the user interface and easy to configure and execute tests. We used to manage our own test cases in the same model. However, it is not open enough. It is not easy to develop two times above. You Need To Ask spirent to send someone to assist and train. The spirent test kits are all written in TCl. We think it is a bit difficult for Tcl to develop large test software, and it is difficult to find errors when errors occur. The development environment is not friendly enough, it is not as convenient as C and Java. Python has developed faster than Tcl in recent years, with many libraries available for calling and a friendly development environment. However, both tcl and Python are less efficient. Our self-written test management tool is also compatible with a large number of internal test tools, which can directly run the original case, but only record the test results and logs. In this way, you do not need to rewrite it.

We have studied Teek's testing tools. The graphical design of tech is the most comprehensive. Even designing a test process, you only need to pull it with the mouse. In this way, flexibility is limited, tests can only be designed according to the pattern it sets. Secondary Development is difficult. Its Configuration and data are also stored in its own format and must be opened and modified using its tools. I personally do not like such a test tool.

Our company also has some automated testing tools, some developed by ourselves, and some outsourced. I also don't like these tools very much, because they have weak functions, poor graphics, and inflexible, and cannot keep pace with the times. These codes are all written from the beginning, which is far from the quality or quantity of open source code and public libraries.

By the way, let's talk about the process of developing and testing tools.

To develop a testing tool, you must first understand the needs of testers. Requirements include requirements for testing functions and how to use tools.

Then define the module and interface. The module includes the function module and the input and output module. The functional modules are segmented from top to bottom. The details are determined by the complexity of the functions and the number of interfaces. Each module should be as independent as possible to reduce their dependencies. Draw several data flowcharts to see which interfaces are required and how interface parameters are defined. When defining interfaces, you must consider scalability. Of course, the scalability and simplification are sometimes necessary. We do not have to cover all aspects, but at least we need to consider the recent expansion needs. Find existing library functions, minimize the workload of self-development, and adopt some standard practices as much as possible. The input and output modules should communicate with testers to call as many interfaces as possible to implement complex test behaviors. Input and Output modules are often the key to the success or failure of a testing tool, which is often overlooked by software personnel. For performance testing tools, hardware Affordability must be considered. If we can achieve distributed operation, it is of course the best.

The development process of the test tool is best with the tester pair-work, so that the tester can continuously use the tool for feedback. The tester is the user of the test tool, and the voice from the user is always the most important. The degree of automation is negotiable, and the automation of Test Result Analysis is often thankless. Automation should focus on configuration, execution, and management. Handle errors with caution. Try not to stop the test if an error occurs. This should be weighed by the testers.

In fact, there is no much difference between the development of test tools and the development of common software. It seems that there is not much help to do with it. But many companies are facing issues about how to develop and use their own testing tools.