Automated website Testing System-system functions

In the previousArticleAutomated website testing system-based onSeleniumAndVstt,Data-Driven Testing,In the testCodeHard-coded Test DataDescribes the process of writing test code. However, writing the code is not enough. Automated Testing not only automates the originally manually executed test cases by coding. A complete automated testing process should include the following processes:

1. Automatically select a machine in the lab to perform the test process. For example, to test a software product Office For example. Because Microsoft already Windows Released many versions, Windows XP, Windows Vista , Windows 7 And Windows There are also different variants, such Vista There are enterprise, flagship, and professional editions. Each variant provides a slightly different function. In addition, you must consider 64 Bitwise AND 32 Bit operating system installation, as well as international testing and so on. In this way, in order to fully test Office The test matrix must be very large. You may need to test hundreds of platforms. Even if each platform requires a test machine, hundreds of machines are required. It is not only time-consuming and labor-consuming to manage these machines manually, but also has a high chance of errors. - For example, you cannot find one. 32 Bit processor machine Installation 64 Bit operating system. Therefore, a large software product team can use or write their own tools to manage the testing machines.

 

The same is true for testing websites. Considering the compatibility between browsers, different testing types require different testing machines, as well as international factors, many machines are also required to perform testing. Because the production of this software involves some concepts of distributed development, I will not explain how to implement this system in this series of articles.

 

2.Automatically prepare the test environment. Since the machine has been selected from the test machine cluster, the next step is to prepare the test environment, such as reinstalling the system (of course,GhostCan also be restored), install the software on which the product depends, and install the latest version of the product (the new version generated after daily compilation), automate the test casesProgramCopy to the test machine and prepare test data.

 

3.Execute automated test cases. This process includes the process we generally understand that coding is used for manually executed test cases to make them run automatically. In addition, this process also includes some optional sub-processes:

 

A)It is not easy to generate random and legal test data automatically based on the test data required for the test case. Although you canRandom ()Such functions generate random data, but using this simple method is difficult to generate legal data. For example, to test the user logon system of a website, most websites require that the user name cannot contain special characters. In this way, you need to add some restrictions during the random data generation process.

 

Generally, software products have different restrictions when receiving user input. Therefore, it is not easy to provide a common code library that generates random but legal test data. This is why, in this series of articles, I will introduceData-Driven Testing.

 

B)Automatically generate automated test cases. This step does not mean that the recording test step is generated according to the test step.C #Or code in other languages. Here, the software automatically generates test cases and the corresponding automated test code. For example, unit tests generally design test cases based on function parameters. In programs, parameter types are generally limited, orProgramming LanguageThe built-in fixed type is either a programmer-defined type. In this way, we have the opportunity to automatically generate test cases based on the parameter type.PexIt is a good example in this field.

 

For example, a tester can design a product model, that is, describe the functions that the product should implement, and then analyze the model to generate a test case.Spec ExplorerThis is an example.

 

For example, if you are testing a function library, for example. NET Framework. Generally, when a user (programmer) uses a function library, it is nothing moreAPI. For each publicAPIDesign the unit test code and write a program to randomly combine these unit test cases to generate new test cases. For example, assume that you want to test the database connectionAPI, AccordingOpen, readdata, closeWrite unit test cases for functions, and then the program will randomly arrange these cases into some new cases. Of course, the random sorting problem is that an invalid call sequence is generated, suchClose, readdata, openThis sequence is invalid. Therefore, a very important task of randomly generating programs in test cases is to remove these illegal sequences with the cooperation of test engineers.

 

4.Automated test result collection because testing cases are executed on multiple machines at the same time, requiring testers to manually collect test results is a very troublesome process; therefore, it is necessary for the program to automatically complete this part, Generally, after the test cases are executed, the automated test script Automatically releases the test results to a central database. The project management team uses some report services-For exampleSQL Server Reporting ServicesSystem to evaluate the following content:

 

A)Components of the product are highly risky, that is, they are prone to errors or have not been fully tested.

B)Product robustness.

C)Can I publish a product or postpone the release?

This article roughly summarizes the tasks that should be completed by the automated testing system, which should have been written as an introduction. However, I think many people may not be interested in purely theoretical things, so I wrote some implementation details first.

To be continued ......