Best practices in mobile app testing

Daniel Knott has used a variety of different programming languages and software quality assurance tools. He has been working on software development and testing for seven years and since 2010 he has been working for Xing AG in Hamburg, Germany, and several projects, such as the Xing Survey and Xing advice, are responsible for test management, test automation, and test execution. Daniel is now the head of the quality assurance team for the Xing Mobile and Xing API team. In the Xing Mobile team, he is also responsible for Xing test management and test automation for Android and iphone apps. Daniel has extensive experience in software testing automation including tools like Robotium, KIF (Keep It functional), Selenium and Java. He also made presentations at various agile conferences and posted them regularly on his blog and Xing blog.

When it comes to software testing, testers are basically thinking about checking files, features, APIs, performance and determining whether the software is secure, and other things about the specific parts of the software. For mobile testing, testers have to consider mobile-related functions based on the user's mobile usage patterns.
This article is based on my work experience and written. As a software Quality assurance Manager for an Agile software development team, I am dedicated to mobile apps and mobile web Apps for iphone, Android, Windows Phone 7. In the daily work of the Xing Mobile team and in communicating with other mobile testing experts, I have a deep understanding of the difficulty of mobile testing. Gradually, I made clear what is a mobile best practice that helps improve my colleagues and my testing efforts and provide users with higher-quality apps.

　　 Functional Testing
Each new feature of the development needs to be tested. Functional testing is an important aspect of mobile app testing, and mobile testers should perform manual and automated tests. At the beginning of the test, the tester must manually test the mobile app as a "black box" to see if the functionality provided is correct and works as designed. In addition to classic software testing, like clicking a button to see what happens, testers must also perform more versatile mobile device-specific testing.
Today, modern mobile devices have touch screens that require multi-touch actions to interact with them. The device can be a portrait or landscape display. They provide motion, tilt and helical sensors. They have different interfaces to connect to other devices or services, such as GPS,NFC, cameras, LEDs, and so on.
The mobile software tester must ensure that all of the app's specific device features are available in the app. There are so many types of mobile devices that it is not possible to test all of the coverage, so the testers have to focus on the key to their app when it comes to functional testing. What is really simple and effective? Device rotation. I found a lot of bugs during my test work that only needed to rotate the device from portrait to landscape and back.
In addition to the entire manual testing process, test automation is also important for mobile apps. Each code change or new feature can affect existing features and their status. Usually the manual regression test time is not enough, so the tester has to find a tool to perform automated regression testing. There are many mobile test automation tools available, commercial and open source, for different platforms such as Android,iphone,windows Phone 7, BlackBerry and mobile web apps. Depending on the development strategy and structure, quality assurance experts need to identify the automation tools that best suit their environment.
Android words, there is robotium[rob01], robolectric [ROB02], Roboguice [ROB03], Monkeytalk [Mon01],monkeyrunner [MON02], Nativedriver [ NAT01] and Calabash for ANDROID[CAL01] and other open source tools. The automation tool Robotium has become the real standard in the open source world. It is simple to use and is based on an Android test device.
Test automation Tools for the iphone include Kif (Keep It functional) [Kif01],uiautomation [UIA01], Monkeytalk [MON01], Calabash for IOS [Cal02],frank [FRA01], zucchini [Zuc01] and so on. All of these tools can also simulate real user interaction on the device or iOS emulator. Choosing a tool is not easy for test automation, but one thing to keep in mind when making a decision is because it is important that test automation should use the same programming language as the product code. If the test and the product code are written in the same language, it's good for both the tester and the developer, because it makes it easier for them to do pairing code. Testers can communicate with developers at the same level, and they can perform code reviews of tests and product codes. For test automation, developers can write their own scripts in the language they are accustomed to.

　　Summarize:
?? Test the app as a "black box" and try to break it.
?? Open each screen of your mobile app and turn your device from portrait to landscape and back to portrait.
?? Don't forget to test device-specific features, such as sensors and communication interfaces.
?? Write test automation scripts for mobile apps.
?? Choose a test automation tool that adapts to your company's policies and structure.
?? The test and product code should be in the same language.

non-functional testing
Another important aspect of mobile app testing is the non-functional requirements of mobile apps. Mobile testers have many questions to test before they launch the market or make further development.
The first test to be performed during the early development phase should be a usability test. This is usually done by an alpha user or colleague. Walk into a café or restaurant and ask the people inside about their app usage. Let them look at the first version of the current development and collect feedback to see if the user can use the new feature well in order to get a first impression.
Check the app's performance. Compare the released version to the current version to see if performance is the same. Better? Or is it worse? Install the app on your old device to see if the app still works on your old device, whether it's good or bad for your hardware device. The most advanced equipment will do the same.
Test Phone, SMS, MMS, Weibo or other notifications come in when the app reacts. Check your battery when using the app. Make sure the test process test equipment is fully charged and check the battery usage every 10 minutes to see if the app consumes too much power. Install the app on the device at low power to see what happens. Check your app's memory usage. If the app stores data in a local file system, test the usage of different memory cards. Think about what happens when local storage is almost full.--app will crash or pop up an error alert box to notify the user?
Test the installation and removal process for your app. More importantly, test the process of upgrading from an older version to a new version. Perhaps the local database has changed, which can cause some serious migration problems. Is the
app localized? Testers need to test the app in different languages. Remember to test with different network speeds on different web carriers. Make sure the app works in GPRS, EDGE, UMTS, LTE and WiFi environments.
Don't forget to check how the app reacts when the network connection is bad or completely dropped. Use the app in airplane mode to see what happens if a request fails. Connect the test device to your computer and check the development log file for exceptions, warnings, or other strange anomalies. These are just a few of the non-functional requirements that mobile testers and developers should consider when developing and testing an app. It is never possible to check in place on every aspect, so the overall team should support QA members to cover as many aspects as possible to prevent users from getting a bad experience.  

　　Summarize:
?? Do a usability test.
?? Compare the performance of the app's released version and the new version.
?? Check your phone, SMS, MMS or Weibo or the app's response when you come in.
?? Check the power of the test equipment.
?? Test your app's memory usage.
?? Install and remove apps.
?? Test the process of upgrading from an older version to a new version.
?? Check the conversion of the language.
?? Use apps in different vectors and network connections, such as Gprs,wifi, or LTE, in the environment.
?? Check the log file for errors or exceptions.

Best practices in mobile app testing