Lightweight UI test automation with. NET Development

James McCaffrey

Download the code for this article: TestRun0501.exe (131KB)



Content of this page
Applications to be tested
Test automation scripts
Operating the application to be tested
Checking application state
Discuss

Manual user interface Testing is one of the most basic types of software testing that most software engineers have used for the first time. Paradoxically, automated user interface testing may be one of the most technical challenges in the type of test written. The Microsoft®.net environment provides you with a number of ways to write automated user interface test Automation. A common and useful method is to record keystrokes, mouse movements, and clicks, and then play back in the application to ensure that it executes as expected. (For more information on this method, see the Bugslayer column of John Robbins in Msdn®magazine March 2002.) This issue of the Paul dilascia column of MSDN Magazine also illustrates how to use. NET to send this type of input to another application. In this month's column, I'll explore another way to write lightweight UI test automation for. NET applications.

The best way to do this is to start the discussion with a screenshot. Figure 1 shows that I have a virtual application to test. It is a color synthesizer application that allows the user to type a color in a text box control, and then type or select a color in Combobox, click the button, and the Listbox displays a message indicating the result of the two colors "mixed". In Figure 1, red and blue can produce purple according to the application. UI Test Automation is a console application that launches a test form, simulates the user moving the application form, defines and adjusts the size of the application form, sets the value of the text box and the Combobox control, and clicks the button control. Test automation checks the result state of the test application, verifies that the Listbox control contains the correct message, and records the "pass" result. The screenshot in Figure 1 is in the test Automation simulation user Click to close the test application File | Captured prior to Exit.



Figure 1 Form UI test automation


In the following sections, I'll briefly describe the virtual applications I tested, explain how to use the reflection and System.Windows.Forms.Application classes to start the application forms in the test automation program, and describe how to use the System.Reflection The methods in the namespace simulate user actions and check application state, and describe how to extend and modify the test system to meet your needs. I think that no matter what role you play in the software production environment, the ability to quickly write lightweight UI test automation can make your skills much better. In addition, even if you are using an existing framework such as NUNIT, these same technologies can be integrated into your own unit test management and related.

Applications to be tested


Let's take a look at the application to be tested in order to understand the goal of test automation. The color synthesizer application to be tested is a simple Windows® form. The code for the application is written in C #, but I will introduce you to the UI automation technology that applies to any application written in the. NET target language. I accept the Visual studio®.net default control names Form1, TextBox1, ComboBox1, Button1, and ListBox1. Of course, in a real-world application, you should change the name of the control to reflect their functionality. I added three virtual menu items: File, Edit, and View. The code listed in Figure 2 is the main content of the application to be tested.

When the user clicks the Button1 control, the application gets the values in the TextBox1 and ComboBox1 controls. If these two color strings match, a message of this color is displayed. If the text box and the Combobox control contain "red" and "blue" respectively, the result message "purple" is displayed. If the text box and the Combobox control are grouped with any other color, the resulting message, black, is displayed. Because this is just a virtual application for demo, I want to keep the code as short as possible, so I don't check the input parameters as I did in the actual application. Although this application is simple, it has most of the basic features of windows-based applications that are required to demonstrate automated UI testing.

Even for such a small application, it is cumbersome, error-prone, time-consuming, and inefficient to test its user interface manually. You must type some input, click the button control, visually validate the result message, and record the results in an EXCEL spreadsheet or other data store. Because the application accepts the user's free input in the text box control, the possible test input is virtually limitless, so you have to test hundreds or even thousands of inputs to understand the behavior of the application well. For all of these actions, once the application code has changed, you must perform the same manual tests from scratch. Writing unit tests is a better approach because they allow you to simulate the users who use the application and then determine if the application responds correctly.

Back to the top of the page
Test automation scripts


Figure 3 shows the overall structure of test automation management, and Figure 4 shows the code outline. I use C # here, but you can easily modify the code to any. NET based language.



Figure 3 UI Test Automation architecture


I first add and declare references to System.Windows.Forms, System.Reflection, and System.Threading namespaces. Because the console application does not reference System.Windows.Forms.dll by default, you need to add a project reference to the System.Windows.Forms.dll file to use the classes in this namespace. The System.Windows.Forms namespace contains the Forms class and the application class, which are used in this solution. I use the class in System.Reflection to get and set the value of the form control and invoke the method associated with the form. Use the methods in System.Threading to run the startup form from the console application test.

I have declared three class scope objects, because there are several ways to use them in test management:

static Form testform = null;
static BindingFlags flags = BindingFlags.Public |
BindingFlags.NonPublic |
bindingflags.static | BindingFlags.Instance;
static int delay = 1500;

Because the color synthesizer application is just a Windows form, I declare a Form object to represent it. There are many ways to use the BindingFlags object as a filter in the System.Reflection namespace. I set an integer delay variable with a value of 1500 (milliseconds) for the Thread.Sleep method so that it can be paused for 1.5 seconds at every point in the test automation. I use this code to launch the application to be tested:

Console.WriteLine ("\nlaunching WinApp under test");
String exepath = "C:\\formuiautomation\\winapp\\bin\\debug\\winapp.exe";
Testform = Launchapp (ExePath, "Winapp.form1");


I have defined the Launchapp method in Figure 5 and its helper method RunApp. This code has a few lines, but it does a lot of work. Note that for the sake of simplicity, I hard-coded the path to the executable file of the application being tested (in your own test, you might want to use parameters to represent this information to make the test automation more flexible). The Launchapp method accepts the path of the application executable file and the name of the application form, and returns an object that represents the form. Launchapp creates an instance of a Assembly object using the Assembly.LoadFrom static method, rather than by explicitly calling the constructor.

Next, the Assembly.GetType method returns a type that represents the application form, and then I use the Assembly.createinstance method to create a reference to the Test form. Then I started a new thread to actually launch the application form. The Application.Run method starts the message loop on the current thread; because I want to perform work while the form is visible, I need to make application.run run in my own thread so that the loop doesn't stop my process. By using this technique, test automation Console Application management and forms run under different threads but the same process. This way you can make them communicate with each other-that is, test management can send instructions to Windows forms.

Back to the top of the page
Operating the application to be tested


After starting the application to be tested, I simulate the user Action application form. The sample test scenario starts with moving the form and resizing the form, as follows:

Console.WriteLine ("Moving Form");
Setformpropertyvalue ("Location", New System.Drawing.Point (200,200));
Setformpropertyvalue ("Location", New System.Drawing.Point (500,500));

Console.WriteLine ("Resizing form");
Setformpropertyvalue ("Size", New System.Drawing.Size (600,600));
Setformpropertyvalue ("Size", New System.Drawing.Size (300,320));

The Setformpropertyvalue method performs all the work (see Figure 6). I use the Object.GetType method to create a Type object that represents the application form, and then use that object to get the PropertyInfo object that references the form's properties, such as the Location property or the Size property. Once you have an attribute information object, you can use the Propertyinfo.setproperty method to manipulate it. SetProperty accepts three parameters. The top two may be what you want-a reference to the object that contains the property you want to change and a reference to the new property value.

The third parameter is required because some properties, such as the Items property of the Listbox control, are indexed. The move form and the size of the form I'm doing here are actually not related to the test application functionality, but I want to tell you how to do it in case your test scenario needs to be implemented. Also note that I am using the ISynchronizeInvoke interface that the Form class (which is actually its control base class) exposes. You should access only the properties of the control that contains the Form by the thread that owns the control's underlying window handle. For a form to be tested, the thread is the one that originated for running Application.Run. Since my test management runs in a separate thread, I need to marshal access to the properties and methods of the control to the thread, making the control's Invoke method and the InvokeRequired property integrated (both the method and the property are ISynchronizeInvoke Part of the interface). For more information about ISynchronizeInvoke, see the article in the MSDN Magazine February 2003, Ian Griffith: Give Your. net-based application a Fast and responsive UI with multiple Threads.

Now I'm ready to simulate the user type a color in the text box control and choose a color from the Combobox control:

Console.WriteLine ("\nsetting textBox1 to ' yellow '");
Setcontrolpropertyvalue ("TextBox1", "Text", "yellow");
Console.WriteLine ("Setting textBox1 to ' Red '");
Setcontrolpropertyvalue ("TextBox1", "Text", "Red");

Console.WriteLine ("Selecting ComboBox1 to ' green '");
Setcontrolpropertyvalue ("ComboBox1", "SelectedItem", "green");
Console.WriteLine ("Selecting ComboBox1 to ' Blue ')";
Setcontrolpropertyvalue ("ComboBox1", "SelectedItem", "Blue");

I set the textBox1 to "yellow", set it to "red", and then set ComboBox1 to "green" and set it to "blue." All the actual work is done by the Setcontrolpropertyvalue method shown in Figure 7.

I use the Thread.Sleep method to pause test automation to make sure that the application that is being tested is started and running. After you create a type object that represents the application form type, I use the Type.getfield method to retrieve information for the specified field (control) in the Form object. Then call the Fieldinfo.gettype method to get the Type object that represents the control you want to manipulate. Once you have the control object, I can manipulate the control object like the Form object, get the PropertyInfo of the control, and then call the SetValue method. For Setformpropertyvalue, you need to make sure that all of the property changes are made in the correct thread.

Please note that test automation does not directly simulate extremely low-level user actions. For example, automation does not simulate the keystrokes for the TextBox1 control, but instead sets the Text property directly. Similarly, automation does not simulate clicking on a ComboBox1 control, but instead sets the SelectedItem property. This is a design flaw in my test automation system. To do that kind of testing, you can follow the advice of John Robbins in the article mentioned earlier.

After you simulate a user's action to type a color in a text box and Combobox control, automation simulates the click of a button control:

Console.WriteLine ("Clicking Button1");
InvokeMethod ("Button1_Click", New Object[]{null, eventargs.empty});

I have defined the InvokeMethod method as shown in Figure 8.

InvokeMethod gets the Type object representing the application form to be tested by calling the Object.GetType method. Then I use Type.getmethod to get information about the specified method and call MethodBase.Invoke to execute the specified method. Invoke accepts two parameters. The first is the form instance to which you want to invoke the method, and the second is an array of arguments for the method. For a button control click Method, the signature looks like this:

private void Button1_Click (object sender, System.EventArgs e) in order to meet the parameters of the Button1_Click method, I need to pass an object representing the sender and an optional event data EventArgs object. For button clicks, I ignored the value of the first parameter, although for the actual test system, I should pass the control as the sender that caused the method to be invoked (the implementation of the method might depend on access to the control if the event handler method is used as a handler for multiple controls). This information is particularly useful). For the second argument, I pass an empty EventArgs object.

Note that test automation simulates a button click by directly calling the relevant method of the Button control, rather than by triggering an event impersonation. When the actual user clicks the button, it generates a Windows message that is processed by the control and converted to a managed event. This event causes a specific (or set) method to be invoked. So if the application associates the wrong method with the button click event, the UI test automation does not catch a logical error (although each test fails and you quickly discover the problem). This problem can be corrected by getting the underlying multicast delegate that uses the reflected event, and then using the delegate's Getinvocationlist method when the event is raised to get a list of all the delegates to invoke. You can then invoke each delegate individually. Alternatively, you can use the EventInfo of the event and its Getraisemethod methods to get the MethodInfo of the method that raised the event, but only a custom throw method is returned, and the Microsoft language that supports the custom throw method has only C + + and Microsoft Intermediate Language (MSIL). Again, all of these issues can be avoided by using the Send Keys method discussed earlier.

Back to the top of the page
Checking application state


After automation sets the state of an application form by simulating user typing and clicking, you can check the system state to see if the application responds correctly (see Figure 9).

I set a Boolean variable named "Pass" and set it to true-I assume that the application's state is correct and check that state, and set pass to False if there is an error somewhere. Check the TextBox1 control to make sure that its Text property is set to "red" correctly. Then check to make sure that the ComboBox1 value is "blue" and that ListBox1 displays the correct message "result is purple". If all checks are passed, the pass message is printed, otherwise the fail message is printed.

The key to checking the state of the application system is the Getcontrolpropertyvalue method I coded, as shown in Figure 10. First use Object.GetType to create a Type object that represents the application form. Then use Type.getfield to get the information for the specified control. Then use GetType to get the Type object that represents the control. Finally, use GetProperty to get information about the specified property of the control and use the GetValue method to get the control property value. GetValue requires an Index object parameter because the attribute can be indexed (for example, when I try to get the Items property of the Listbox control).

Note that it is more flexible to check the text in the ListBox1 control than to check the text in the TextBox1 control. I use my Getcontrolpropertyvalue method to access the Items property and then check it using the Contains method.

After checking application status and recording pass or fail results, I can easily exit the application being tested:

Console.WriteLine ("\nclicking menu file->exit in 5 Seconds ... ");
Thread.Sleep (3500);
InvokeMethod ("Menuitem4_click", new object[] {null, New EventArgs ()});

Although the application is terminated when test management terminates, because they are running in the same process, and the application being tested runs in a background thread, it is best to explicitly exit the application through test management in order to explicitly clean any allocated system resources.

Back to the top of the page
Discuss


If you want to automate UI testing before. NET is available, there are actually only two options. First, buy commercially available UI automation tools. Second, use the Microsoft Active accessibility (MSAA) API to create your own UI automation tools. The system I'm introducing is a good complement to the other two strategies. There are several excellent commercial UI automation tools available for you to use. The advantage of these tools is that they are fully functional. The downside is that you need to pay for them, have a steep learning curve, and do not allow you to access the source code when you need to modify the feature. Using MSAA gives you complete control over your automation tools, but it takes a long time to learn. In fact, in several projects I've done, MSAA UI test automation is likely to be as complex as a test application!

The automated UI test method I mentioned here has been successfully used in several large and medium sized products. Because it can be implemented quickly and easily, it can be used early in the product cycle when the system being tested is unstable. However, because the UI test system is relatively lightweight, it cannot cope with all possible UI test scenarios. You can modify and extend this design in a variety of ways. Because the intention of this presentation was to give the reader a rudimentary understanding, I removed most of the error checking and hard-coded Most of the information to make it clearer and simpler. It is necessary to add a lot of error-checking code to test automation-after all, you expect to find errors.

Depending on your production environment, you can parameterize some parts of the test system. In terms of test terminology, the system I refer to is called a test scenario-a series of operations that control the state of the application to be tested (unlike a test case), which usually refers to a smaller operation, such as passing a parameter to the method and checking the return value. For example, to parameterize the scenario, you can create an input file that looks like this:

[Set State]
TextBox1:Text:yellow
TextBox1:Text:red
ComboBox1:SelectedItem:green
ComboBox1:SelectedItem:blue
Button1:button1_click
[Check State]
TextBox1:Text:red
ComboBox1:SelectedItem:blue
ListBox1:Items:Result is purple

You can then have your tests automate reading, parsing, and using the data in that file. You can also enter data as a test scenario using either XML or SQL. The test system I described records its results in the command shell. You can easily redirect the results from the command line to a text file, or rewrite automation to directly record the results.

The next generation of Windows (code-named "Longhorn") will introduce a new representation subsystem code-named "Avalon". Avalon will refer to the UI Test Automation concept I introduced to a higher level. We need to provide platform-level support for the automation of all UI elements and expose a consistent object model for all user controls. This allows developers and testers to quickly and easily create extremely powerful UI test automation. The technology in this column is a reminder of how revolutionary Avalon works.

Before. NET, writing automation is often a resource-intensive task, and test automation (especially UI test automation) is often placed at the bottom of the product task priority list. But with. NET, you can write very powerful test automation with just a fraction of the time you've spent before.


Please send the questions and suggestions you want to give to James to testrun@microsoft.com.

James McCaffrey is in charge of the Volt Information Sciences, Inc., and is responsible for technical training for software engineers working at Microsoft. He is involved in developing a variety of Microsoft products, including Internet Explorer and MSN Search. You can get in touch with James via jmccaffrey@volt.com or v-jammc@microsoft.com.