Best practices in mobile app testing

Daniel Knott has used a variety of different programming languages and software quality assurance tools. He has been working on software development and testing for seven years and since 2010 he has been working for Xing AG in Hamburg, Germany, and several projects, such as the Xing Survey and Xing advice, are responsible for test management, test automation, and test execution. Daniel is now the head of the quality assurance team for the Xing Mobile and Xing API team. In the Xing Mobile team, he is also responsible for Xing test management and test automation for Android and iphone apps. Daniel has extensive experience in software testing automation including tools like Robotium, KIF (Keep It functional), Selenium and Java. He also made presentations at various agile conferences and posted them regularly on his blog and Xing blog.

?

When it comes to software testing, testers are basically thinking about checking files, features, APIs, performance and determining whether the software is secure, and other things about the specific parts of the software. For mobile testing, testers have to consider mobile-related functions based on the user's mobile usage patterns.
This article is based on my work experience and written. As a software Quality assurance Manager for an Agile software development team, I am dedicated to mobile apps and mobile web Apps for iphone, Android, Windows Phone 7. In the daily work of the Xing Mobile team and in communicating with other mobile testing experts, I have a deep understanding of the difficulty of mobile testing. Gradually, I made clear what is a mobile best practice that helps improve my colleagues and my testing efforts and provide users with higher-quality apps.

　　 Functional Testing
Each new feature of the development needs to be tested. Functional testing is an important aspect of mobile app testing, and mobile testers should perform manual and automated tests. At the beginning of the test, the tester must manually test the mobile app as a "black box" to see if the functionality provided is correct and works as designed. In addition to classic software testing, like clicking a button to see what happens, testers must also perform more versatile mobile device-specific testing.
Today, modern mobile devices have touch screens that require multi-touch actions to interact with them. The device can be a portrait or landscape display. They provide motion, tilt and helical sensors. They have different interfaces to connect to other devices or services, such as GPS,NFC, cameras, LEDs, and so on. 
The mobile software tester must ensure that all of the app's specific device features are available in the app. There are so many types of mobile devices that it is not possible to test all of the coverage, so the testers have to focus on the key to their app when it comes to functional testing. What is really simple and effective? Device rotation. I found a lot of bugs during my test work that only needed to rotate the device from portrait to landscape and back. 
In addition to the entire manual testing process, test automation is also important for mobile apps. Each code change or new feature can affect existing features and their status. Usually the manual regression test time is not enough, so the tester has to find a tool to perform automated regression testing. There are many mobile test automation tools available, commercial and open source, for different platforms such as Android,iphone,windows Phone 7, BlackBerry and mobile web apps. Depending on the development strategy and structure, quality assurance experts need to identify the automation tools that best suit their environment. 
Android words, there is robotium[rob01], robolectric [ROB02], Roboguice [ROB03], Monkeytalk [Mon01],monkeyrunner [MON02], Nativedriver [ NAT01] and Calabash for ANDROID[CAL01] and other open source tools. The automation tool Robotium has become the real standard in the open source world. It is simple to use and is based on an Android test device. 
Test automation Tools for the iphone include Kif (Keep It functional) [Kif01],uiautomation [UIA01], Monkeytalk [MON01], Calabash for IOS [Cal02],frank [FRA01], zucchini [Zuc01] and so on. All of these tools can also simulate real user interaction on the device or iOS emulator. Choosing a tool is not easy for test automation, but one thing to keep in mind when making a decision is that it is important: Test automation should use the same programming language as the product code. If the test and the product code are written in the same language, it's good for both the tester and the developer, because it makes it easier for them to do pairing code. Testers can communicate with developers at the same level, and they can perform code reviews of tests and product codes. For test automation, developers can write their own scripts in the language they are accustomed to.

　　Summarize:
?? Test the app as a "black box" and try to break it.
?? Open each screen of your mobile app and turn your device from portrait to landscape and back to portrait.  
?? Don't forget to test device-specific features, such as sensors and communication interfaces.  
?? Write test automation scripts for mobile apps.  
?? Choose a test automation tool that adapts to your company's policies and structure.  
?? The test and product code should be in the same language.

　　 non-functional testing
Another important aspect of mobile app testing is the non-functional requirements of mobile apps. Mobile testers have many questions to test before they launch the market or make further development.
The first test to be performed in the early stages of development should be a usability test. This is usually done by an alpha user or colleague. Walk into a café or restaurant and ask the people inside about their app usage. Let them look at the first version of the current development and collect feedback to see if the user can use the new feature well in order to get a first impression. 
Check the app's performance. Compare the released version to the current version to see if performance is the same. Better? Or is it worse? Install the app on your old device to see if the app still works on your old device, whether it's good or bad for your hardware device. The most advanced equipment will do the same. 
Test phone, SMS, MMS, Twitter or other notifications come in when the app reacts. Check your battery when using the app. Make sure the test process test equipment is fully charged and check the battery usage every 10 minutes to see if the app consumes too much power. Install the app on the device at low power to see what happens. Check your app's memory usage. If the app stores data in a local file system, test the usage of different memory cards. Think about what happens when local storage is almost full.--app will crash or pop up an error alert box to notify the user? 
Test the app's installation and removal process. More importantly, test the process of upgrading from an older version to a new version. Perhaps the local database has changed, which can cause some serious migration problems. 
Is the app localized? Testers need to test the app in different languages. Remember to test with different network speeds on different web carriers. Make sure the app works in GPRS, EDGE, UMTS, LTE and WiFi environments. 
Don't forget to check how the app reacts when the Internet connection is bad or completely dropped. Use the app in airplane mode to see what happens if a request fails. Connect the test device to your computer and check the development log file for exceptions, warnings, or other strange anomalies. These are just a few of the non-functional requirements that mobile testers and developers should consider when developing and testing an app. It is never possible to check in place on every aspect, so the overall team should support QA members to cover as many aspects as possible to prevent users from getting a bad experience.

　　Summarize:
?? Do a usability test.
?? Compare the performance of the app's released version and the new version.  
?? Check your phone, SMS, MMS or Weibo or the app's response when you come in.  
?? Check the power of the test equipment.  
?? Test your app's memory usage.  
?? Install and remove apps.  
?? Test the process of upgrading from an older version to a new version.  
?? Check the conversion of the language.  
?? Use apps in different vectors and network connections, such as Gprs,wifi, or LTE, in the environment.  
?? Check the log file for errors or exceptions.

　　Test equipment--fragmentation
For a mobile quality assurance person, the key question about mobile testing devices is, "What is the best tool for testing?" "This problem has to be solved because it can't be tested on every device!" In this regard, there are two major players in the mobile device market: Android and ios! But for geographical reasons, some other platforms are also used. There are Windows,blackberry, WebOS, SymbianOS, and function machines. The table in figure I shows the use of the smart-machine operating system provided by Chinese, German, and U.S. suppliers.

Figure one: Intelligent machine Operating system
Data Source: www.thinkwithgoogle.com/mobileplanet/en

Almost every platform has different vendors selling smart machines with different hardware, software specifications and custom user interfaces. such as Android, there are like Samsung, HTC, ASUS, LG, Motorola, Sony, Huawei and other suppliers. This is an important example of device fragmentation, and it's really hard to find the right test equipment. Moving a webpage is another pretty difficult issue, because there are too many mobile browsers, such as: Safari, Opera Mini,dolphin, Android and RIM native, Google Chrome, Firefox, Internet Explor Er9 and other feature machine browsers! So what's the test equipment to choose? Just use the latest browser version? To buy every kind of equipment on the market? or using an emulator?
Small bet the simulator here: Don't test with simulator! They may be useful for basic testing, but the results are different from the results on the real machine.
In my opinion, a good idea to solve this test device problem is to combine the device with the browser. For example, mobile testers can combine devices based on their hardware and software specifications. Each combination determines a priority, such as a= highest, b= average, c= lowest. Each group contains devices that are assigned to that category based on the platform and vendor. 
　　  Overview of possible combinations:
?? Group 1, priority c:cpu and small devices with small ram and low resolution. Legacy software versions and browsers 
?? Group 2, Priority B: General CPU, RAM <512 MB, display size and resolution good mid-range devices. The software is not up-to-date.
?? Group 3, priority a: dual-core/four-core CPUs, RAM >512 MB, high-resolution devices. The latest software version.
The three groups cover the vast majority of users on a particular platform and represent the other phones that fit this group on the market. This can reduce the work required during development and testing.

　　Summarize:
?? Combine and select the preferred test device and browser version.
?? Do not test with the simulator.

　　Combo Tool
As mentioned before, mobile testers must automate the testing of mobile apps to ensure that code changes do not affect current functionality. Another best practice is to combine test tools and set them up as a continuous integration server to execute them from the center. Developers need to write unit tests for their code to ensure that each small component is working as expected. In addition, end-to-end inspection testing using tools like robotium or keep It functional is useful as a user.

　　Summarize:
?? Combine the test tools and set them into a continuous system.

　　Internal Beta version
If a mobile team wants to communicate with the beta testers of a mobile app earlier, they can create their own in-house App Store, like Android and iphone. With the Hockeykit [HOC01] tool, the team can pass the new version of the app to colleagues via the company's WiFi. This is an effective way to get important feedback from colleagues, especially if the team or tester doesn't have a chance to show the app to the outside world. Hockeykit also provides useful data on how to test the app and what OS versions and devices are used by colleagues. It also includes a crash reporter to see the cause of the current development version errors and crashes.

　　Summarize:
Get early feedback with an internal beta release.

　　Understanding Customers
Mobile testing is more effective if developers and test teams understand that the app's customers will be used in the future. A good place to collect customer information is the App store for a specific vendor (Apple App Store, Google Play store, Windows marketplace, etc.). If a team already has an app in these stores, they get information about the devices, software versions, languages, and vectors that customers use. Figure 2 Below is an example of the Google Play store that shows data for a published Android app. Instead of mobile Web pages, there is no app store to provide relevant user data. Therefore, it makes sense to collect information about the "User Agent" (device, browser) used in mobile Web pages. Aware of this, the team can optimize the various devices and software versions and reduce the effort spent on developing and testing them.

? Figure 2:google Play store data on the Android device and OS version used

In addition to user data, customer reviews in the App store need to be carefully collated to gather important feedback and their expectations of new features and respond to reported bugs.

　　Summarize:
?? Understand the devices and software versions your customers use
?? Leverage data from the supplier market
?? Take the reviews in the App Store seriously

　　Take the initiative in the Forum community
This is the last best practice, which is not directly related to routine mobile testing, but may help you think from a different perspective and get new ideas.
Be an active software tester in the community, like this:
?? Utest (www.utest.com)
?? Mobileqazone (www.mobileqazone.com)
?? Softwaretestingclub (www.softwaretestingclub.com)

In such a community, software testers can exchange views with other (mobile) QA experts on a variety of test topics. They can share knowledge and help other people solve their problems. Testers can get great ideas when writing test cases: How to write good bug reports, and also improve their own test and test automation skills.

Copyright notice: This article from SPASVO Software Testing network: http://www.spasvo.com/news/html/2014112095134.html

Original works, reproduced when you must be in the form of hyperlinks to the original source, author information and this statement, or will be held liable.

?

?

Best practices in mobile app testing