Automated Software Testing (5)-How to Establish a testing scenario?

Test scenario(Test scene), you can also understand it as a test environment. It is a scenario where automated test cases (ATC) are executed. We can define it as follows: test scene is the prerequisite for the ATC to execute, and is independent of the application under test, AUT ). What we will discuss here is, what method should be used to establish test scene is more appropriate?

Gui automation test requires that we use mouse clicks and keyboard input as much as possible to implement the GUI test method to meet the test requirements that truly match the end-user. For example, if we want to test whether a dialog can be correctly disabled from the GUI automation test point of view, you should do it like an end-user, click Close button/cancel button (if any) instead of send wm_destroy message. Otherwise, if our programmers do not implement the corresponding function of the close button, the dialog is obviously disabled, but the automation test script does not find this bug as imagined.

However, it would be inappropriate to put the same view on the establishment of test scene. I can at least find the following reasons:

1. Creating Test scene is a necessary process of GUI automation test, but it is not the ultimate goal. Our ultimate goal is to test and verify AUT;
2. Base 1) item, you do not need to test the test scene;
3. The efficiency of using GUI test method is not high, which is greatly affected by GUI, And the maintainability of script is poor;
4. Compare againest 3) item, using non GUI test method to establish test scene, high efficiency, better versatility, and no lengthy script;

For example, our test is an ActiveX-Web Test Based on IE. It is clear that different IE Security Settings will pop up in different dialog boxes when this ActiveX-web is opened. My Recommended Practice is: if the test case does not have special requirements, set the IE uniformly. It belongs to the scope of test scene. You can use non-Gui test method to modify the value of IE in the registry for the purpose of setting. If the test case has such requirements, it is clearly pointed out that when the IE Security is set to medium or low, the corresponding warning box appears when ActiveX-web is opened. You can use non-Gui test method to create this special test scene. Of course, you must also add some test scripts for verification and processing for the warning box.

To sum up my example, we can draw a conclusion: In GUI automation test, try to use the non-Gui test method to establish test scene, and try to use the GUI test method to test aut. If you still have questions, think about the ActiveX-web ATC of IE, we do not seem obligated to perform GUI test for Microsoft's settings on IE, do we?

Austin Chen
2009-3-20