On the instability of Web UI Automation test (II.)

There are two levels of instability in Web UI Automation testing: technical-not structured, robust, and stable-running scripts non-technical – project reasons or attempts to achieve inappropriate goals with Web UI Automation, resulting in frequent scripting changes and high maintenance costs

1th in the previous article it from the code level analysis, today, mainly said 2nd.

Ugly words said that in front of theWeb UI Automation was not a balm, it could not kill the manual testers. Measuring good input and output, maximizing the utility of UI Automation is the kingly way.

What projects are not suitable for Web UI Automation testing. I think there is not too much Web UI interface, no end-user-oriented projects, it may only involve background data import, processing, or model calculation, which is more suitable for interface testing, each new development of code into the target environment, let CI automatically run the interface test without UI interface, to see if there is a new bug introduced. In general, automated testing without a UI interface is more efficient and more stable.

How to test this tool with Web UI Automation to find a balance between development maintenance costs & automated outputs.

My advice is: prioritize and automate the core. For all Web projects, legal login & Illegal login, and permission control are the most important and basic. In addition, carry out the core of the system and the user the most frequent use of the part, from the perspective of the function of automated testing, that is, click on this button to determine whether the profile page is aroused; Configure the report, whether the final report generated, whether there is data. (If the data is correct, it should be verified by the integration test); Make a trade, whether the return value is successful, contains exception message, and so on. UI Automation should focus on logical functions, not the UI, and don't let automation really judge the UI, or else it's a trap. Do not be misled by the name, in fact, UI Automation is to simulate the user's mouse operation, to determine whether the system function is abnormal, it can also judge the UI, but this will be dead. Why. Because the Web project changes fast and big change, today this page this control is blue, width 25, tomorrow may become white, width 15, who said the quasi? The more automated code is judged, the higher the maintenance cost, the more unstable. So automation should be focused on: This control is not a very functional function, not a control style. The style of the control should be given to the manual tester to judge visually. In agile projects, it is difficult and unnecessary to achieve UI automation and development synchronization. Because the new function module is impossible to one step, synchronous development of automation code, the subsequent changes, maintenance costs will remain high. So should be partially automated, until the module stability, and then increase the proportion of automation.

A friend of mine asked me "How practical is Web UI Automation." Is it necessary to take time to realize it? 』

My answer is yes. From my personal experience,Web UI Automation testing is very practical and strong. It captures the unexpected bugs of some manual testers. For example development recently in modifying an existing function module A. The manual tester has tested all aspects of the A module, PASS. The automation test a module is also pass, but the B module hangs up. This is because the B module is affected by the change of a module. But these effects are likely to be unexpected, not to mention testers, even to develop themselves. It is not that the testers are inexperienced, because some functional modules have implicit coupling, changed here, testers will not be able to test the entire system again. The entire system is measured once this kind of thing should be given to automation to complete.

Our project will release a new version to the customer about every 8 weeks, the UI Automation test is quite a bit, because it does not live up to expectations, every time before the release of the new version, can always find some strange bugs, these bugs will be developed the first time fix off. Another time when the new version was released, due to bugs found by UI Automation, the entire iteration was postponed, the bugs were fixed before the new version was released and a new iteration started. Incomplete statistics, our automation every month to capture more than a bug, on average every 3 days 1 bugs. and UI Automation is two people. I personally think that our UI Automation from the input-output ratio, is very cost-effective, big Boss should be very satisfied.

Finally, I wish you all a happy National day, enjoy this year's last holiday.