Practical Guide to Mobile end app testing

In this paper, from the point of view of the tester, we put forward more than 100 problems to be considered during the test of mobile app. Whether you're a tester, a developer, a product manager, or an interactive designer, these issues can be of reference value when developing mobile apps. I and queen together translate this text, share to everybody, hope to have help and inspiration.

Original English: http://mobile.smashingmagazine.com/2012/10/22/a-guide-to-mobile-app-testing/

－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－－

Testers are often seen as bug-seekers, but have you ever wondered how they actually conducted the tests? Are you curious as to what they are doing and how they can embody value in a typical technical project?

The authors will take you through the testers ' thinking process and explore their various considerations when testing mobile apps. The purpose of this article is to reveal the thinking process of testers and to show the breadth and depth of what they usually consider.

Testers need to ask questions

The core competency of testers is to raise challenging and related issues. If you can combine research, questioning skills with technology, and product knowledge, you will become a good tester gradually.

For example, testers may ask:

· What platform should this app be used on?

· What exactly does this app do?

· What happens if I do this?

Like。

Testers can discover problems from a variety of scenarios, possibly from conversations, designs, documents, user feedback, or the product itself. There are too many possibilities ... So let's find out!

Where to start testing

Ideally, testers should have all the latest details of the product being tested. But in fact this is rare, so, like everyone else, testers will have to use limited information on their hands. But it's not an excuse for not being able to test! Testers can actually gather information from a variety of internal and external sources.

At this stage, testers can ask these questions:

· What information is available: Spec? Project meeting? User documentation? Knowledgeable team members? Is there a support forum or a company online forum to help? Is there a record of existing bugs?

· What systems, platforms, and devices are the applications operating and testing?

· What type of data does this application deal with (such as personal information, credit cards, etc.)?

· Does the application have consolidated external applications (such as APIs and data sources)?

· Does the application need to use a specific mobile end page?

· How do existing consumers evaluate this product?

· How much time is available for testing?

· What are the priorities and risks of testing?

· Which users are not happy to use, why?

· How do I publish and update?

Based on the information gathered above, testers can develop a test plan. Usually the budget determines the method of testing, and the method of measuring one or one months is certainly different. When you become familiar with the team, the workflow, and the solution to this type of problem, you are more likely to anticipate the results.

Case: Facebook app Social review

When collecting information as a tester, I prefer to use the Facebook app as a case in which users complain everywhere. The following only shows some of the challenges users have commented on in the itunes App Store, and there are many more on the web.

Facebook apps on the iphone have a lot of negative comments

If I take the challenge to test the Facebook app, I would certainly consider the feedback, or it would be a fool.

The creativity of Testers

You may know what this app is supposed to do, but what can it do? How does the user actually use it? Testers are good at thinking as bystanders, experimenting with different things, and constantly asking "if ... "and" why "the question.

For example, testers at the mobile end are often tested with different user roles-certainly a bit exaggerated, but the ability to think, analyze, and visualize themselves as a different user is inspired by the test.

Testers may envision themselves as the following users:

· have no experience;

· Very experienced;

· Lovers；

· Hackers；

· Competitors.

Of course there are more optional roles, depending on what product you are developing. In fact, in addition to role characteristics, its operational behavior and workflow is also very important. The way people use products is often strange, for example:

· Returned when it should not be returned;

· Impatient and repeatedly knocking buttons;

· Enter the wrong data;

· Don't understand what to do

· may not be set as required;

· You may be self-righteous to think you know what to do (for example, you don't usually read the instructions).

Testers often find unexpected bugs when they encounter these problems. Sometimes these bugs are trivial, but more in-depth investigations will reveal more serious problems.

Many problems can be predetermined and tested. When testing the mobile app, the following questions are not all relevant, but you can also try to ask:

· Do you follow what you say?

· Did you finish the task by design?

· Did you finish the task by design?

· What would happen if you were in the position of being used or loaded? Is it unresponsive? Will it crash? Will it be updated? Any feedback?

· Does the crash report feed back to the app?

· What are the creative, logical, or passive ways in which users can navigate? Do users believe in your brand?

· How safe is the user's data?

· Could it be interrupted or cracked?

· What happens when you run to the limit?

· Would you like to open related services (such as GPS, Wi-Fi)? What happens if the user opens? What happens when you don't open it?

· Where to redirect users? To the web? From web to app? Does this cause problems?

· Does the communication process and market feedback conform to the app's function, design, and content?

· What is the login process? Can you log on to the app or go to the end of the page?

· Does the login integrate other services, such as logging in with Facebook and Twitter accounts?

Case: Run Keeper ' s Gy Update

Runkeeper, is a tracking your fitness activities of the app, the latest release has a "goal settings" function, I am interested to experience, part from the tester's point of view, more as a genuine love of the product users to experience. But I found some problems:

1. The default unit is sterling, but I want to weigh the kg as a unit of weight;

2. The switch between sterling and KG is not easy to use;

3. When the goal is set, it leads to the presentation of incorrect data and graphs, which confuses me.

4. Because of article 3rd, I wanted to delete the target, but I could not find a place to delete it;

5. In order to solve this problem, I have to change the value of personal weight, until the "target set" range, so that the goal is achieved, you can reset the target;

6. I will try to add the target again;

Because of the above doubts, I spent more time playing with it, to see if I can find other problems;

Here are screenshots of some of the problems found:

The latest version of the app contains a new "Target" section. When I set the date, I found that the start and end dates start in 1, and why there are two 1 years to choose from: The year column should be shown as "1, 2, 3" from the top down?

Another bug, which is a spelling error in the "Current Weight" section, will have a misspelled "Enter" when the data is emptied (Etner in the application), which is a small bug but looks unprofessional.

There is no shortcut to finding a problem, you can only try it over and over again. Each app and its team will face many different challenges. But the typical trait of a tester is to go beyond the limit, do something unconventional, change things around you, and keep testing for a long time (test days, weeks, or months, not minutes), even when you know it's impossible. These are also the scenarios that can be found and elicited.

Where's all the data?

Testers like to find problems with data, which makes developers sometimes frustrated. In fact, users or software developers are really confusing in the flow of information, because there may be a lot of errors, so services based on data and cloud are more important

Maybe you can try to check out the problem in the following scenario:

· Mobile device data is full;

· The tester removed all data;

· The tester removed the app, what about the data?

· Testers Delete and reload the app, what about the data?

· Too much or too little content leads to changes in design and layout;

· In different time periods and time zones;

· Data is out of sync;

· synchronization interrupted;

· Data updates affect other services (such as Web pages and cloud services);

· Fast processing of data or processing of large amounts of data;

· Use invalid data;

Case: soup.me Error

The soup.me I tried was a Web service that could classify photos in my personal Instagram through maps and colors, but I didn't do it for long. When registering, it prompts me to Instagram on the picture not to be many, but my account has clearly had more than 500 photos. I don't know where the problem is, maybe it's a data problem, it's a presentation, it's probably a problem with the app.

Another case: Quicklytics

Quickytics is a Web analytics application on an ipad. During use, although I have removed the site configuration from Google Analytics, it still exists. Here are some questions:

· I have deleted the site configuration, why still have this information?

· The left module does not explain why "This operation cannot be completed", can it be improved to avoid confusing users?

Testers also like to test the situation under the limit data. They are often used as a typical user to understand the app, so testing at the limit won't take long. The data is confusing, so testers have to take into account the type of software users have and how to test in different data scenarios.

For example, they might try the following scenarios:

· Test the limit value that the user can enter;

· Test with duplicate data;

· Test in a new mobile phone with no data;

· Test on old mobile phone;

· Pre-installation of different types of data;

· Consider gathering everyone's resources for testing;

· Automate some tests;

· Test with some more than expected data to see how it is handled;

· Analyze how information and data affect the user experience;

· Always ask questions regardless of what the user is seeing.

Create error reminders and messages

Here, I'm not talking about the design of a good error message from a designer's point of view, but I want to look at it from the point of view of the user or the tester. Error reminders and messages are places where testers can easily spot problems.

Questions to ask about error messages:

Please consider the following questions:

· Can the UI design for error reminders be acceptable?

· is the error message content understandable?

· Does the error message remain consistent?

· Are these error messages helpful?

· is the error message content appropriate?

· Are these errors consistent with conventions and standards?

· Are the error messages inherently secure?

· Can running records and crashes be available to users and developers?

· Have all the errors been tested?

· What state will be in when the user finishes processing the error message

· Is there an error message that pops up when the user should accept the error message?

The error message affects the user experience. However, bad or useless error reminders are everywhere. Although the ideal state is to prevent users from encountering error messages, this is almost impossible. The design, implementation, and validation of error situations may be contrary to expectations, but testers are often adept at discovering unexpected bugs and are careful to refine them.

Case of error message

I like the example of a Facebook app on the iphone. These lengthy and obscure words not only attempt to cover many different scenarios, but may also be lost without any reason.

May the following message box be included in the counter case "Hall of Fame"?

Look at the Guardian app on this ipad, what if I don't want to "try again"?

Considerations on specific platforms

It is critical for any project team member to understand the business, technical, and design constraints of the relevant platform.

So what are some of the platform-related issues that testers of mobile apps should identify?

· Have you followed the design specification for this particular platform?

· How does it compare with competitors and the design in the industry?

· Are you comfortable with peripherals?

· Touch screen support gestures, such as: Pat, double click, long Press, drag, shake, pinch, flick, slide?

· Can this app be understood?

· What is the change in the direction of the device?

· Can I use maps and GPS?

· Do you have a user's guide?

· is email workflow friendly?

· Does it run smoothly when you share it across the Web? Are other social applications or sites integrated?

· Is it still working when the user is multitasking and switching between different apps?

· Does it show time progress when the user updates it?

· What is the default setting?

· Would it be different to use sound effects?

Case: Chimpstats

Chimpstats is the ipad's last application for viewing details of email ads. The first time I use this application is in the horizontal screen mode. I was trapped when I needed to enter the API password. I can't enter the API password in the horizontal mode until I switch to the vertical screen mode to enter the success.

Problems connecting and interrupting a lot of interesting things can happen when the connection is intermittent or accidentally interrupted.

Have you ever tried to use the app in the following scenarios:

· Moving around?

· Wi-Fi connection?

· Without Wi-Fi?

· 3G mode?

· Intermittent connection?

· Set to Flight mode?

· When a call came in?

· When you receive a message?

· When you receive a reminder notification?

· When the battery is low or even shuts down automatically?

· When you are forced to update?

· When you receive a voice message?

This type of test is the easiest to find errors and bugs. I strongly recommend that you test in these situations (not just turn on the boot, make sure it works, try the entire process that the user uses, and force connections and interrupts within a specific time interval).

· Does this app provide enough feedback?

· Is the data transfer known to the user?

· Will it slowly stop and then crash?

· What happens when you open it?

· What happens when the task is done?

· Is it possible to lose unsaved operations?

· Can you ignore notification reminders? What happens when you ignore it?

· Can you respond to notification reminders? What happens when you respond?

· Is it appropriate to use error messages for certain issues?

· What happens when a login expires or times out?

App Maintenance

The process of trying to speed up the whole test is simple, just test it once and for all, right? Please think twice.

One of the problems I've encountered at the moment is that some apps on the ipad are not available to download after they are updated. This is very frustrating for a user.

Maybe that's something that developers can't control. Who knows? I only know that it is not available to users. I also tried uninstalling the app and then reloading it, but the problem was never resolved. I have a lot of searches on the web, and there's no other way to fix it than to find some suggestions for updating the operating system. Maybe, I'll try again next time.

The key problem is that if an application is tested once and only once (or tested in a very short period of time), there are a lot of problems you won't find. An app itself may not find change, but outside conditions can make these problems happen.

When the external environment continues to change, how will the app be affected? Let us ask ourselves:

· Can I download this app?

· Can I download and install updates?

· Can you use it after the update?

· Can I update a lot of apps when they are waiting to be updated?

· What happens when the system is updated?

· What happens if the system is not updated?

· Will it automatically sync downloads to other devices via itunes?

· Does it make sense to automate tasks or tests?

· Does it connect to a network service? What difference does it make?

After each release of the app on the mobile end, it's best to test it. Each time a new version is released, the highest priority test is defined to ensure that it can be performed under various conditions (primarily on the mainstream platform). Over time, tests can become automated. But keep in mind that automation is not a panacea and that problems can only be found through human eyes.

Case: Analytics Application on iphone

I've been using this app for two years and it hasn't been a problem. But now, it shows me some Web site data is 0 (but actually, more than one person has visited my site within one months!) ）。 Judging from the App Store commentary, I'm not the only one who's met the problem.

Another case is Twitter on the iphone. After updating and launching the app, I instantly saw this prompt: "Your timeline data is empty and you're not paying attention to anyone" (but I'm an active user with 5 years of experience). I was worried for a moment, and thankfully, the news quickly disappeared and then loaded out of historical data.

Testing is not a right or wrong judgment.

We discussed some aspects of mobile testing, but the premise is that problems can be found with problems.

Typically, tests are considered fully logical, predictable, and predictable, including test scripts and test plans, pass and fail, and correct and erroneous feedback. Walking through these test processes is not far from the truth.

Of course, if necessary, we can use the above methods to test, but this is not the purpose of the test. Not only are we trying to create test cases, finding bugs, but more importantly, finding the key issues that will provide valuable information for the team to decide when to release the app. The best way to find those key questions is to ask questions!

To understand the "CDC turn off", please go to: "CDC turn over" to hit the customer! Fanke is coming! (Link: http://cdc.tencent.com/?p=6518)

(This article is from the Tencent CDC Blog, reprint, please indicate the source)