Test run: Customizing Test automation with Team System

The best way to test your software is more than one. In addition to manual testing, depending on your development environment, you can use the Business test automation framework, open source and internal test automation frameworks, and custom test automation scripts. All of these methods have their advantages and disadvantages.

The advantage of customizing your test automation scripts is that they are quick and flexible to write. However, manageability is the bottleneck for customizing test automation. Large-volume test scripts, test case data, and test results overwhelm the test. Fortunately, you can use the Visual studio®2005 team System to manage custom test automation. I'll use some screenshot to explain this. First, consider the execution test automation script shown in Figure 1.

Figure 1 Typical custom test automation scripts

Test automation is a very short JavaScript script that performs a module test on a method named Trimax that returns only the largest of three integers. The Trimax method resides in a typical COM DLL file. Although simple and effective, there are several drawbacks to testing automation using this type of method. Where will the test results be stored? This test run is based on which version of the DLL is executing? How can I share these test results with other members of the development team? Are these results related to any public error? The Visual Studio team System deals specifically with this type of problem, as shown in Figure 2.

Figure 2 Using Team System to manage custom test automation

Visual Studio Team System consists of a client version of Team Foundation Server (TFS) and Visual Studio. You can view TFS as an intelligent back-end data warehouse that stores and manages all the data associated with a software development project, including source code, errors, test results, specification documents, and other content.

The custom test automation features I covered in this column are available in the Visual Studio team Edition for testers. You can download the 180-day evaluation version of the Visual Studio team Suite (which includes all versions of functionality) and the team Foundation Server from Microsoft.com/downloads.

I'll show you three examples in this column. The first example shows how testers use Vste to extract a very simple custom test automation and encapsulate it in tests that can be managed by Team System. The second example goes a step further, showing you how to modify the custom test automation so that you can create test case results (with more detailed information than just passing or failing to advertise). My third example shows how to use the team Foundation Server to store and manage test run results data.