Introduction to Mobile GUI Automation test (including Android and iOS)

Summary

As we all know, automated testing can reduce the burden of testers to a certain extent, improve test efficiency, and achieve reliability testing and performance testing through Automation. For mobile client testing, it would be cool if we could get the phone to run the application automatically to help us detect the correctness of the function. Youdao Test Group on a number of popular mobile phone automation tools were investigated, and selected a number of tools for practical use. This paper will introduce the research and implementation of the Mobile client (Android&ios) GUI automation tool based on the practical work. Android Tools

The instrumentation class provided by the Android APIs [1] can initialize the Android application code, allowing you to monitor the application's system interactions, cooperate with KeyEvent, motionevent classes, and send user events to automate the GUI layer. The test program needs to inherit ACTIVITYINSTRUMENTATIONTESTCASE2 to automate.

To facilitate the writing of automated test cases, we need to extend the ActivityInstrumentationTestCase2. The industry also has some mature automation tools, such as Robotium, Athrun, Nativedriver, Monkeyrunner and so on. We need to focus on the needs of our own products, choose a suitable tool to automate. For the automation of mobile client GUIs, it is necessary to ensure that the tools selected have the following characteristics: Tools open source, easy to expand. Script writing is concise and maintenance cost is low. Meet the automation requirements of the client. and verify the correctness of the results. can be used for continuous integration.

Table 1 lists the differences between the four tools:

	Robotium	Athrun	Nativedriver	Monkeyrunner
Scripting languages	Java	Java	Java	Python
Easy to expand
Easy to maintain				0
Meet demand	0			0
Easy to check				0
Sustainable integration				0

Table 1 Comparison of Android automation tools

Monkeyrunner by writing Python scripts to automate, the results are validated by a screenshot of the picture to achieve, the verification method is not flexible, not recommended.

Nativedriver [2] is an implementation of the Webdriver interface, using the mobile client native UI instead of the browser UI (Selenium) automated test tool. Similar to the Selenium RC way to run the test program, for those who are familiar with Webdriver, get started quickly. As you can see from table 1, this tool can also meet our automation requirements, but in the early stages of the research, the tool provides fewer interfaces to meet test requirements and is now well advanced. No use of this tool can be considered historical reasons.

Robotium [3] is well known, based on instrumentation implementations, to provide the interface to meet most of the automation requirements. But do not support WebView, and Youdao dictionary search results show just using the WebView component, the module there is no way to automate through the tool.

The implementation of Athrun [4] is similar to Robotium, provides a lot of interfaces, and supports WebView, and also enables continuous integration. For our products, to meet the needs of automation. Because the tool is open source, we can also do some encapsulation on the basis of the tool when the demand is not satisfied in the future. So finally chose Athrun to implement the GUI automation of notes and dictionaries. instance

Like writing Android apps, you first need to create an Android test project that specifies the Android project that is being tested. If the source code is not applied, you can also modify the package of the application under <instrumentation> tags under the androidmanifest.xml file of the test program targetpackage for us to test. After you import Framework.jar, you can start writing automation scripts. Figure 1 is an automated use case for the Youdao Cloud Note Android Login module:

Figure 1 Android Automation Use Cases

We need to inherit athruntestcase, specify package and the activity we want to start. Each method acts as a test case, and finally runs the test case in the form of the Android JUnit test. One thing to note here is to install the tested application on the device first. If the application is signed, then we need to use the same signature for our test application.

As you can see from the code, the interface for the component is very simple and can be found by attributes such as ID, value, and so on. If a component does not have a relatively independent attribute, it can also be looked up through the parent node of the component. The Hierarchyviewer tool provided by the Android SDK presents the UI components of the simulator's current activity as a tree, with a clear view of the detailed attributes of each component.

There are also a number of ways to verify that a test case is passed, to verify that the activity jumps, to display the components, and, of course, that some of them may need to pass through the screenshot. Figure 1 Determines whether a test case is passed by validating the jump of the activity after the execution of the use case.

Above we have completed a test case automation. Obviously based on the Athrun framework, our automated use case authoring is much more convenient and costs are greatly reduced.

In order to complete a variety of automated test cases, it is necessary for us to have a more detailed understanding of Athrun so that we can develop the required interfaces two times. Interested friends can download the tool source code through Http://code.taobao.org/svn/athrun/trunk/android/. IOS Tools

Before introducing iOS automation, first introduce the instruments [5] tools provided by Xcode. The tool is a practical tool for dynamically tracking and analyzing OS X and iOS code. This is a flexible and powerful tool that allows you to track one or more processes and check the data collected. In this way, instruments can help you better understand the behavior of the application and the operating system.

Instruments in addition to providing automation tools automation, there are tools such as leaks, allocations to detect and analyze program performance. For iOS testing and development, tracking and locating defects, instruments is very useful. Figure 2 is the main interface of the instruments:

Figure 2 Instruments Main interface

Interested friends can learn about the use of various tools provided by instruments in the IOS Developer Library.

The next step is to introduce the UI Automation, which can be used in a real device or emulator to perform automated tests.

Use JavaScript to write test cases and invoke UI Automation interfaces to simulate user interaction. The tool returns the log information for the automation run to the instruments Information Bar. instance

Start instruments in Xcode by profile. Alternatively, double-click the start instruments to select the application tested on the device in the instruments main interface. Then write the automation scripts in the Automation script bar. One thing to note about using iOS devices is to make sure the developer Profile setting is release mode. Figure 3 is an automated use case for the Youdao Cloud notebook iphone Login module:

Figure 3 IOS Automation use case

After you complete the automation script, you can run it by clicking on the Instruments Record button, or by instruments related commands. The former is suitable for debugging, and the latter is suitable for continuous integration.

As you can see from Figure 3, the corresponding UI components need to be specified on a per-layer basis. There are two ways to locate these components. For systems above iOS 5, you can record the build script, but the recording may log errors. So in order to locate the exact component location, you need to call Logelementtree, print the hierarchy tree structure of the current screen component to the log, and locate the component through the log.

As you can see from the diagram above, the validation mechanism provided by the UI Automation is cumbersome, and verifying that each component exists requires judgment. This is just the tip of the iceberg, UI Automation will be caught pop-up (alert) Click the Default Action button, provided by the log file is also not easy to analyze, so it is necessary for the UI Automation interface two times encapsulation, more convenient to write automation use cases.

Athrun also provides an automated framework for iOS, with three implementations. The first kind of Appframwork is code injection type, need to insert test code in the source, objective-c write test case, automation cost is high, do not recommend to use. The second kind of instrument athrun is to extend the interface of UI Automation, which improves the stability of the original interface operation. The third type of instrumentdriver is based on the Instrument JS framework to develop the Instrumentdriver server, implement the client on Java, and use Java Script to control iOS automation execution. The framework also implements a single step running, debugging, and other UI Automation features. Figure 4 is the Instrumentdriver architecture diagram:

Fig. 4 Instrumentdriver Frame composition

From the introduction of Instrumentdriver, we can see that there are many advantages over the UI Automation provided by Apple. At present, because we need to use automated scripts with instruments other performance testing tools to achieve performance testing of the application, it is still the use of instruments native tools with extended JS script. Later, if you can find a better way to get performance data, you will gradually turn to the instrumentdriver way to automate iOS. contrast

From table 2 You can look at the differences between the iOS and Android Automation tools we've chosen:

	Scripting languages	Test a single page	UI component Positioning	Test results
Ios	Javascript	Not supported, starting from view start automation	Recording Logelementtree	Not easy to Jenkins [6] show, need to transform format
Android	Java	Specifies the corresponding activity when inheriting	Hierarchviewer	JUnit run results for easy Jenkins display

Table 2 Comparison of iOS and Android Automation Tools

The implementation of automation to some extent improve our testing efficiency, as the Internet product iterations faster, here we recommend automation from the smoke test, use case design to be as concise as possible, to do more encapsulation, in order to reduce maintenance costs. At the same time, the continuous integration of automation, early intervention testing, early detection of problems. On top of that, we can also implement performance testing of the application by automating the script to get performance data. These efforts will allow us to cover a wider range of tests, more testing, better detection and monitoring of product quality. reference materials

[1] Android APIs https://developer.android.com/guide/components/index.html

[2] Nativedriver http://code.google.com/p/nativedriver/

[3] Robotium http://code.google.com/p/robotium/

[4] Athrun http://code.taobao.org/p/athrun/wiki/index/

[5] IOS Library http://developer.apple.com/library/ios/navigation/

[6] Jenkins http://jenkins-ci.org/

[7] from http://techblog.youdao.com/?p=571