Lightweight UI testing automation developed with. net

Download the code in this article:Testrun0501.exe(KB)

Content on this page

	Application to be tested
	Test automation script
	Operate the application to be tested
	Check Application Status
	Discussion

Manual User Interface testing is the most basic software testing type. Most software engineers use this type for the first time. In conflict with this, Automated User Interface testing may be one of the most technically challenging test types written. The Microsoft. NET environment provides you with many automated methods for writing automated user interfaces. A common and useful method is to record the keys, move the mouse, and click, and then play back in the application to ensure that it is executed as expected. (For more information about this method, seeMsdnMagazineThe bugslayer column of John Robbins in June March 2002. This phaseMsdn magazineThe Paul dilascia column of explains how to use. Net to send this type of input to another application .) In this month's column, I will explore another way to compile lightweight UI testing automation for. NET applications.

The best way is to start with a screen snapshot. Figure 1 shows that I have a virtual application to be tested. It is a color synthesizer application that allows users to type a color in the text box control, and then type or select a color in ComboBox. click the button to display a message in ListBox, indicates the result of the two colors "mixed. In Figure 1, red and blue generate purple Based on the application. Ui testing automation is a console application that starts a form to be tested, simulates a user's mobile application form, defines and adjusts the size of the application form, and sets the value of the text box and ComboBox control, click the button control. Test automation checks the Result Status of the test application, verifies that the ListBox control contains the correct message, and records the "pass" result. The screen snapshot in Figure 1 is captured before the test automation simulation user clicks close test application file | exit.

Figure 1Automated form UI Testing

In the following sections, I will briefly introduce the virtual applications I have tested and explain how to use reflection and system. windows. forms. the application class starts the application form in the test automation program and introduces how to use system. the methods in the reflection namespace simulate user operations and check the application status, and describe how to expand and modify the test system to meet your needs. I think that no matter what role you play in the software production environment, the ability to quickly write lightweight UI testing automation can greatly improve your skills. In addition, even if you are using an existing framework (such as nunit), these same technologies can be integrated into your own unit test management and related.

Application to be tested

Let's take a look at the application to be tested to understand the purpose of testing automation. The color synthesizer application to be tested is a simple windows form. The application code is written in C #, but the UI automation technology I will introduce to you is applicable to applications written in any. Net-oriented language. I accept the default Visual Studio. NET control names form1, textbox1, combox1, button1, and listbox1. Of course, in actual applications, you should change the control name to reflect their functions. I added three virtual menu items: file, edit, and view.Figure 2The Code listed in is the main content of the application to be tested.

When you click the button1 control, the application obtains the values in textbox1 and combox1 controls. If the two color strings match, messages of this color will be displayed. If the text box and ComboBox controls contain "red" and "blue", the result message "Purple" is displayed ". If the text box and ComboBox controls are any other color combinations, the result message "black" is displayed ". Because this is only a virtual application for demonstration, I want to keep the code as short as possible, so I didn't check the input parameters as I did in the actual application. Although this application is very simple, it has most of the basic features of Windows-based applications needed to demonstrate automated UI testing.

Even for such a small application, it is cumbersome, error-prone, time-consuming, and inefficient to manually test its user interface. You must enter some input, click the button control to visually verify the result message, and record the result to an Excel spreadsheet or other data storage. Because the application receives free input from the user's text box control, in fact, the test input is infinite, so you must test hundreds or even thousands of input to better understand the behavior of the application. Once the application code changes, you must perform the same manual test from the beginning. Writing unit tests is a better way, as these tests allow you to simulate users who use the application and then determine if the application responds correctly.

Back to Top

Test automation script

Figure 3 shows the overall structure of automated test management,Figure 4Shows the Code outline. Here I use C #, but you can easily change the code to any. Net-based language.

Figure 3Ui test automation Structure

First, I add and declare references to the system. Windows. Forms, system. reflection, and system. Threading namespaces. Because the console application does not reference system by default. windows. forms. DLL, so to use the class in this namespace, you need to add the system. windows. forms. DLL file project reference. The system. Windows. Forms namespace contains the forms class and application class, which are used in this solution. I use the class in system. Reflection to obtain and set the value of the Form Control and call methods related to the form. Use the method in system. threading to start the form from the console application test management.

I declare three scope objects, because there are several methods to use in Test Management:

static Form testForm = null;static BindingFlags flags = BindingFlags.Public |                             BindingFlags.NonPublic |                            BindingFlags.Static | BindingFlags.Instance;static int delay = 1500;

Because the color synthesizer application is just a Windows form, I declare a form object to represent it. There are many methods in the system. Reflection namespace to use the bindingflags object as a filter. I set an integer delay variable with a value of 1500 (MS) for the thread. Sleep method, so that 1.5 seconds can be paused at all times of the test automation. I use this code to start the application to be tested:

Console.WriteLine("/nLaunching WinApp under test");string exePath = "C://FormUIAutomation//WinApp//bin//Debug//WinApp.exe";testForm = LaunchApp(exePath, "WinApp.Form1"); 

I have definedFigure 5The launchapp method in and its helper method runapp. The number of lines in this code is small, but it is very useful. Note that for simplicity, I hardcoded the path to the executable file of the application to be tested (in your own test, you may want to use parameters to indicate this information to make the test automation more flexible ). The launchapp method accepts the path of the executable file of the application and the name of the application form, and returns an object representing the form. Launchapp uses the Assembly. loadfrom static method to create an instance of the Assembly object, instead of explicitly calling the constructor to create it.

Next, the Assembly. GetType method returns a type indicating the Application Form. Then I use the Assembly. createinstance method to create a reference to the test form. Then I initiate a new thread to actually start the application form. Application. the run method starts a message loop in the current thread. Because I want to execute work when the form is visible, I need to make the application. run runs in your own thread so that the loop will not stop my process. By using this technology, application management and forms in the test automation console will run in different threads but in the same process. In this way, they can communicate with each other-that is, test management can send commands to Windows Forms.

Back to Top

Operate the application to be tested

After the application to be tested is started, I simulate the user operation on the application form. The example Test Scheme starts from moving the form and adjusting the size of the form, as shown below:

Console.WriteLine("Moving form");SetFormPropertyValue("Location", new System.Drawing.Point(200,200));SetFormPropertyValue("Location", new System.Drawing.Point(500,500));Console.WriteLine("Resizing form");SetFormPropertyValue("Size", new System.Drawing.Size(600,600));SetFormPropertyValue("Size", new System.Drawing.Size(300,320));

The setformpropertyvalue method executes all the work (seeFigure 6). I use the object. GetType method to create a type object that represents the application form, and then use this object to obtain the propertyinfo object that references the form attribute (such as the location attribute or size attribute. Once you have an attribute information object, you can use the propertyinfo. setproperty method to operate it. Setproperty accepts three parameters. The first two may be what you want-to reference the object that contains the property to be modified and to reference the new property value.

The third parameter is required because some attributes (such as the items attribute of The ListBox control) are indexed. What I have done here to move the form and change the size of the form is not actually related to the test application function, but I want to use it to tell you how to implement it in case your test solution needs. Note that I am using the isynchronizeinvoke interface exposed by the Form class (actually its control base class. You should only access the properties of the control through the thread that owns the control's underlying window handle (including form. For the form to be tested, this thread is initiated to run application. Run. Because my test management is run in a separate thread, I need to mail access to the properties and methods of the control to this thread, make the control's invoke method and invokerequired attributes integrated (this method and attribute are part of the isynchronizeinvoke interface ). For more information about isynchronizeinvoke, seeMsdn magazineIan griith's article in November February 2003: Give Your. Net-based application a fast and responsive UI with multiple threads.

Now I want to simulate a user typing a color in the text box control and selecting a color from the ComboBox control:

Console.WriteLine("/nSetting textBox1 to 'yellow'");SetControlPropertyValue("textBox1", "Text", "yellow");Console.WriteLine("Setting textBox1 to 'red'");SetControlPropertyValue("textBox1", "Text", "red");    Console.WriteLine("Selecting comboBox1 to 'green'");SetControlPropertyValue("comboBox1", "SelectedItem", "green");Console.WriteLine("Selecting comboBox1 to 'blue'");SetControlPropertyValue("comboBox1", "SelectedItem", "blue");

I first set textbox1 to "yellow", then set it to "red", then combox1 to "green", and then set it to "blue ". All practical work is causedFigure 7The setcontrolpropertyvalue method shown in.

I use the thread. Sleep method to pause test automation to ensure that the application to be tested has started and runs. After creating a type object that represents the Application Form type, I use the type. getfield method to retrieve the information of the specified field (control) in the form object. Call fieldinfo. GetType to obtain the type object of the control to be operated. Once a control object exists, I can operate on the control object like a form object, that is, obtain the propertyinfo of the control, and then call the setvalue method. For setformpropertyvalue, make sure that all attribute changes are made in the correct thread.

Note that testing automation does not directly simulate extremely low-level user operations. For example, automation does not simulate the keys of the textbox1 control, but directly sets the text attribute. Similarly, automation does not simulate clicking on the combox1 control, but sets the selecteditem attribute. This is a design defect in my testing automation system. To perform the test in that way, follow the suggestions in the previous article by John Robbins.

After you type the color in the text box and ComboBox controls, the system automatically simulates the click button control:

Console.WriteLine("Clicking button1");InvokeMethod("button1_Click", new object[]{null, EventArgs.Empty} );

I have definedFigure 8The invokemethod method shown in.

Invokemethod: Call object. GetType to obtain the type object that represents the form of the application to be tested. Then, I use type. getmethod to obtain information about the specified method and call methodbase. invoke to execute the specified method. Invoke accepts two parameters. The first is the form instance for which you want to call the method, and the second is the parameter array of the method. In this case, the signature is as follows:

Private void button#click (Object sender, system. eventargs E) to meet the parameter requirements of the button#click method, I need to pass an object that represents the sender and an eventargs object that represents the optional event data. For Button clicking, I ignore the value of the first parameter, although for the actual test system, I should pass the control as the sender who calls this method (the implementation of this method may depend on access to the control, this information is particularly useful if this event handler method is used as a handler for multiple controls ). For the second parameter, I pass an empty eventargs object.

Note that test automation simulates Button clicking by directly calling the related methods of the button control, rather than triggering event simulation. When you click a button, a Windows message is generated. The control processes the message and converts it to a hosted event. This event causes a specific (or a group) method to be called. Therefore, if an application associates an error method with a button-Click Event, UI test automation does not capture logical errors (although each test fails and you will soon find the problem ). This problem can be corrected by obtaining the underlying multi-channel broadcast Delegate of the reflected event, and then using the delegate's getinvocationlist method when the event is triggered to obtain the list of all delegates to be called. Then each delegate can be called separately. Alternatively, you can use eventinfo of the event and its getraisemethod method to obtain the methodinfo of the method for triggering the event, but in this case, only one custom Triggering Method is returned, the Microsoft language that supports custom triggering methods is only C ++ and Microsoft intermediate language (msil ). Once again, all these problems can be avoided by using the send keys method discussed earlier.

Back to Top

Check Application Status

After automating setting the state of the application form by simulating user input and clicking, you can check the system status to check whether the application responds correctly (seeFigure 9).

I set a Boolean variable named "pass" and set it to true-I assume that the application status is correct and check the status. If an error occurs somewhere, set pass to false. Check the textbox1 control to ensure that its text attribute is set to "red" correctly ". Check to make sure that the value of combox1 is "blue", and listbox1 displays the correct message "result is purple ". If all the checks pass, the pass message is printed; otherwise, the Fail message is printed.

The key to checking the application system status is the getcontrolpropertyvalue method encoded, as shown in figureFigure 10. First, use object. GetType to create a type object that represents the application form. Then, use type. getfield to obtain the information of the specified control. Then, use GetType to obtain the type object that represents the control. Finally, use getproperty to get the information of the specified property of the control, and use the getvalue method to get the property value of the control. Getvalue requires an index object parameter because the attribute can be indexed (for example, when I try to obtain the items attribute of The ListBox control ).

Note that checking the text in the listbox1 control is more flexible than checking the text in the textbox1 control. I use my getcontrolpropertyvalue method to access the items attributes, and then use the contains Method for check.

After checking the application status and recording the pass or fail results, I can easily exit the application to be tested:

Console.WriteLine("/nClicking menu File->Exit in 5 seconds . . . ");Thread.Sleep(3500);InvokeMethod("menuItem4_Click", new object[] {null, new EventArgs()} );

Although applications to be tested run in the same process and the applications to be tested run in background threads, however, to explicitly clear any system resources allocated, it is best to exit the application explicitly through test management.

Back to Top

Discussion

If you want to automate UI testing before. NET is available, there are actually only two options. First, purchase commercial UI automation tools. Second, use the Microsoft Active accessibility (msaa) API to create your own UI automation tool. The system I introduced provides two other policies. There are several excellent commercial UI automation tools available for you. These tools provide complete functionality. The disadvantage is that you have to pay for them, there is a steep learning curve, and you are not allowed to access the source code when you need to modify the function. Msaa gives you full control over your automated tools, but it takes a long time to learn. In fact, in several projects I have worked on, msaa-based UI testing automation may be as complex as the application to be tested!

The automated UI testing method I mentioned here has been successfully used in several large and medium-sized products. Because it can be implemented quickly and easily, it can be used early in the product cycle when the system to be tested is unstable. However, this UI testing system is relatively lightweight, so it cannot cope with all possible UI testing situations. You can modify and expand this design in multiple ways. The intention of this introduction is to give readers a preliminary understanding. To make it clearer and simpler, I deleted most error checks and hardcoded most of the information. It is necessary to add a lot of error check code to test automation-after all, what you expect is to discover errors.

Based on your production environment, you can parameterize some parts of the test system. In terms of test terminology, the system I mentioned is called test scenario-a series of operations that control the state of the application to be tested (different from test case, the latter usually refers to a small operation, such as passing a parameter to the method and checking the return value .) For example, to parameterize the scheme, you can create an input file as follows:

[set state]textBox1:Text:yellowtextBox1:Text:redcomboBox1:SelectedItem:greencomboBox1:SelectedItem:bluebutton1:button1_Click[check state]textBox1:Text:redcomboBox1:SelectedItem:bluelistBox1:Items:Result is purple

Then, you can allow your tests to automatically read, parse, and use the data in the file. You can also use XML or SQL as the input data for the test scheme. The test system I introduced records its results into the command shell. You can easily redirect results to text files through command lines, or rewrite automation to directly record results.

The next-generation windows (codenamed "Longhorn") will introduce a new representation subsystem codenamed "aveon. Avron will introduce the UI test automation concept I introduced to a higher level. We urgently need to provide platform-level support for automation of all UI elements, and to publish consistent object models for all user controls. This allows developers and testers to quickly and easily create extremely powerful UI test automation. The technology in this column is a reminder of the revolutionary way of working aveon.

Before. net, coding automation is usually a resource-consuming task, while testing automation (especially UI testing automation) is usually placed at the bottom of the product task priority list. However, with. net, you only need to spend a small part of the previous time to compile a very powerful test automation.

Please send your questions and suggestions to JamesTestrun@microsoft.com.

James McCaffreyWorking at volt Information Sciences, inc., is responsible for technical training for software engineers working at Microsoft. He is involved in the development of a variety of Microsoft products, including Internet Explorer and MSN Search. You can useJmccaffrey@volt.comOrV-jammc@microsoft.comContact James.

Go to the original English page