Android self-initiated test solution

Today, there are a large number of Android self-motivated test architectures or tools available for us to use, including Activity instrumentation, Monkeyrunner, Robotium, and Robolectric. In addition, Lesspainful also provides services to carry out their own active testing on real equipment.

Android itself provides basic support for the instrumentation test, One of them is the ActivityInstrumentationTestCase2 class in the Android.test package, which expands the JUnit TestCase class to provide a functional test of the Android activities. In the application test, each activity is first initialized by instrumentation and then loaded into the Dalvik virtual machine of the Android emulator or device.

The Android SDK comes with a test tool Monkeyrunner, which provides the API and execution environment to execute test code written in the Python language. It provides APIs to connect devices, install/uninstall apps, perform applications, take screenshots, and compare images to infer whether the screen after a particular command executes includes expected information and performs the corresponding test. Monkeyrunner using ActivityInstrumentationTestCase2, Providertestcase,servicetestcasek, Singlelaunchactivitytestcase and other classes to define test examples and use the Instrumentationtestrunner class to perform tests.

Robotium is also a instrumentationtestrunner to complete the Android Interactive test architecture, which spans multiple activities, support function test, System test and receive test. Robotium supports activities, Dialogs, Toasts, Menus, Context Menus, and even honeycomb, and it can be integrated with Maven and ant for continuous integration testing. Robotium is called another selenium for Android apps.

Robolectric, which does not rely on the testing capabilities offered by Android, uses the shadow objects and performs a test on a normal workstation/SERVERJVM, unlike a simulator or device that requires dexing (Android The Dex compiler compiles the class file into the format used by the Dalvik VM on Android devices, and the process of packaging, deploying, and executing significantly reduces the time it takes to test execution. The pivotal lab claims that using Robolectric can perform 1047 tests in 28 seconds.

Lesspainful has taken the Android test a step further, providing a multi-device platform for its own proactive testing service. The user uploads the application (*.apk) and the test file written in cucumber (a business-related DSL), selects the device configuration required for the test execution, and finally tests its own initiative and generates a test report. The devices it supports include Garmin Asus, several Htc,lg,samsung galaxy,sony Xperia and Motorola Motodefy.

In order to understand the service details provided by many other lesspainful, we visited Lesspainful CEO Jonas Maturana Larsen. Here is a brief interview:

InfoQ: What's wrong with executing applications on different version numbers of Android? To ensure that the program executes properly, does the developer need to test his application on every version of Android?

JML: For example, a bug in SAXParser before Android 2.2 exists in a callback to Contenthandler.startelement, which causes the application to produce the wrong behavior.

So far, we have found the difference between different operating system version numbers in very many ways. Some of these may cause crashes on 2.1-update1, but can be performed normally on 2.1-update3 and 2.2.

InfoQ: Is there a real difference between devices for Android? Can you give us a sample, for example, that the Android2.2 app is able to perform on HTC but not on Samsung? (or any other combination of Android version numbers and device manufacturers)

JML: on LG phones, Horizontalscrollviews sometimes causes the background picture on the child view to disappear. This problem exists in all of the LG phones we tested, no matter what the Android version number is.

Assuming you don't handle this type of problem yourself, it will cause your app to vary across devices. For example, Motorola will highlight an input field with a red border. In one of the projects I used to participate in, we used the same red border to indicate that the input was wrong.

Other issues, rather than manufacturers, are related to hardware: for example, some phones use smaller RAM and high-resolution cameras, which can cause crashes when you process images on your phone.

InfoQ: How do these tests work?

JML: testing is like executing a ActivityInstrumentationTestCase2, mostly using Robotium. The only change we make is to remove an existing signature, and then generate our signature file again for it.

Once the test execution is complete, the app will be uninstalled and the phone will be restored to its original settings.

InfoQ: What are the advantages of the services you offer compared to Monkeyrunner,robotium and robolectric?

JML: Lesspainful is a service, not just an architecture. We want to create a service that not only makes testing possible, but also saves us a lot of testing time and can help us find many other bugs than any other architecture.

In addition, we believe that the use of cucumber can clearly define high-level test descriptive narratives that can be better shared by people outside the development team at the same time.

In Git, for example, we want to be like GitHub, not just the usual git libraries.

InfoQ: Do you have any plans to support many other devices in the future?

JML: yes. We plan to continue to add support for many other devices. Assuming this requirement is raised, we will try to finish it.

For now, we're also working on support for iOS devices, and hopefully the beta version will be released this fall.

InfoQ: What is lesspainful Enterprise Edition?

JML: We will provide a toolset that is similar to a Mac Mini, but we are flexible enough to meet the needs of our customers. Lesspainful Enterprise Edition is not yet officially launched, so please look forward to it.

English Original: Automated testing Solutions for Android