Mobile app Test Learning (1)

The most common app Bug, crash test case design

Mobile app testing has a certain amount of complexity compared to traditional desktop testing.

These complexities can be categorized as:
Environment (large number of devices, various mobile OSS, adaptable to frequent OSS changes).
devices (touch and non-touch devices, limited memory capacity, battery power consumption).
Network (different networks and operators, in the case of bad or no network app behavior, offline support).
Usability (direction, touch, multi-touch, zoom, paging and navigation limitations, various distractions such as incoming calls, incoming SMS, alarms, and low battery alarms).
All of these phone-specific complexities require new test case designs for mobile app testing.

According to the results of the survey, mobile app crashes are the most common mobile app Bug, which is expected as a result because it's easy to spot a mobile app crash.

The Android OS has a pop-up screen with a "forced shutdown error" on it;

When a crash occurs, the app screen in iOS disappears suddenly. In the worst case, app crashes can cause system failures and operating system crashes.

mobile app crashes cause  
Why mobile apps often crash. There are several reasons for app crashes: from platform or environment to development issues.  
Some crash causes (in no particular order):  
device Fragmentation : Apps may behave differently on different devices due to the diversity of devices. &NBSP
Bandwidth limit : A network with poor bandwidth may not have enough fast response time for apps.  
Network changes : Switching between different networks may affect the stability of the app.  
Memory Management : Low available memory, or use of an unauthorized memory location may cause the app to fail.  
Too many users : Too many connections can cause the app to crash. &NBSP
Code Error : A new feature that has not been tested may cause the app to fail in a production environment.  
third-party services : Ads or pop-up screens may cause the app to crash.

test case design for mobile app crashes
Test cases are one of the most important parts of mobile testing.
Preparing and executing pre-defined test cases for mobile app crashes will simplify and speed up testing for mobile app crashes.
Some common test scenarios that trigger the crash of the mobile app are as follows:
1 Verify app behavior on multiple devices with different screen resolutions, operating systems and operators.
2 Verify the app's behavior with the newly released operating system version.
3 Verify app behavior in environments such as tunnels, elevators, etc. that suddenly change the quality of the network.
4 change from cellular to Wi-Fi via manual network, or in turn, verify app behavior.
5 Verify app behavior in a network-free environment.
6 app behavior when verifying incoming/SMS and device-specific alerts, such as alerts and notifications.
7 Verify app behavior by changing the direction of the device in different view modes.
8 Verify the app behavior when the device is low on memory.
9 Verify app behavior by applying loads with test tools.
10 Verify app behavior in different supported languages.

One important principle is to test the version of the app that you end up publishing to your users.

The idea of a daily build and daily test is pervasive, and most of the time we test the app development and debug versions, not the final release version.

When packaging the final release version, we typically add a digital signature, or add code confusion. There must be inconsistencies between the final release version and the Debug version.

Our iphone app used a third-party open-source library that worked perfectly when the debug version was in use, but it was not until it was officially online that it was sure to cause a crash.

This cost and experience is invaluable (in fact, this issue has been discussed and warned in this open Source Library forum). We later spent a lot of effort to correct and remedy the problem.

If you have tested the release version at the outset, this problem does not occur.

There are three very important best practices for testing web-related apps.

1, 2G, 3G, WiFi must be covered

These three are not only the network speed difference, they represent three different network environment.

In addition, you may not have thought of a particular situation that could be used to measure the problem: the development environment and the production environment.

An experienced development team will build a test environment in the intranet to test the development, and switch the configuration to a production environment on line when it is online.

This switchover should be a link in the release process that requires a check. However, we may miss it.

So this test case can be used to prevent the occurrence of this situation, in the WiFi environment can work fine, but 2G and 3G is not, only the real environment 2G and 3G to normal operation (think 2G and 3G can normally access http://192.168.1.xxx such address on it).

2, HTTP, HTTPS to overwrite

Many apps and back-end services are interactive via HTTP, and normal conditions are normal.

Why you need to test your HTTPS environment. In some free internet environment, such as McDonald's, Starbucks, their network environment is to enter a user name and password, through SSL authentication to access the network. If you use the HTTP client's library to not capture this exception, your app will definitely crash.

3, network anomalies, server downtime or 404, 502 and other cases of testing

The stability of the backend service is sometimes difficult to control, especially when it comes to DNS and space service providers.

A well-known domestic DNS service providers often appear large-scale domain name resolution failures, in this case, your request to the background API will likely to appear 404 error.

And the data you interact with the API should be some sort of fixed format, such as JSON and XML, so your data parsing will inevitably have errors and throw exceptions.

If you do not handle the exception correctly, it may cause the program to not work correctly. Here's a pseudo-code to explain the logic:

try {
if (request () = = success) {
    callsuccess ();
} else {
    callfail ();
}
Hidepopup ();
} catch (E) {
    //do nothing, just wait....now popup window would show forever on the!!!
    If it is a IOS app, the popup window would lock the screen
}

1. Functional Testing:

Write test cases according to the product requirements document.

Software design document writing use case.

Note: Test cases are written according to the product requirements document.

2. Compatibility test:

Compatibility with Android version

Phone resolution compatibility

Network compatibility: 2g\3g\4g\wifi, weak network, off-network

App cross-version compatibility

(1) Suitability test:

1> Phone different resolution support: client-supported resolution, etc.

2> phone different version of the support: 2.34.04.4, etc. in the test plan: you need to schedule a separate time for the compatibility testing of different Android systems, including the following version 2.0 and more than 4.0

3>: Different manufacturers of mobile phone manufacturer system support, different manufacturers will have different Android systems, such as: Millet, Huawei, hammer on the market mainstream mobile phone support

4>. Phone different size support: 3.5 to 5.0 screen in UI display is different, to support maximum to minimum.

(2) Installation and unloading test:

1>: Build apk file can be installed and uninstalled on the real machine;

2>. Android phone-side universal installation tool. such as: Pea pod

(3) Online upgrade test:

1>. Verifying digital signatures

2>: Can be used normally after upgrade.

3>: Online cross-version upgrade.

3. Performance test:

Stress test:

Power Flow test:

CPU, memory consumption:

App startup duration

Crash rate

Memory leaks

4. Network test:

1) Outside the network testing the main reality of the simulation of customer use of the network environment, the inspection of the customer single program in the actual network if the environment and the use of business operations.

2) Outside the network testing is mainly covered by WIFI\2G\3G\4G,.NET\WAP, telecommunications, mobile unicom, all possible combinations to test.

Principle:

1) as full coverage of user scenarios as possible, the test case needs to include various combinations of network permutations of possibilities.

2) There is also a time when the analog signal is blocked. The impact of the client and so on. And to do the outsourcing scene test, in the Alpine, hilly, train and other special conditions for comprehensive testing

5. Interface Test:

Client-side and service-side interaction

Client-side data updates and service-side data consistency

The client side was updated when it was disconnected.

The service side was hung when the client side was updated.

6, Business logic test:

1) Business logic test: The main test client business can be completed normally.

2) function Point test: Main test client function point is normal use

3) Relevance test: The main test client and PC-side interaction, after the client processing, the PC side and the client data consistent

7. Abnormal test:

1) Interactive anomaly test: The client as a mobile phone characteristics testing, including the situation of being disturbed, such as incoming calls, SMS, low battery test, but also pay attention to the mobile phone hardware, such as: Standby, Plug and unplug data lines, headphones and other operations will not affect the client.

2) Abnormal test: Mainly includes broken network, power outage, server anomalies, such as the situation, the client can be normal processing, to ensure the correctness of the data.

test cases for mobile-side installation packages

Test item	To test the child	Enter a description	Expected results	Note	Actual results	Test results
Testing of the mobile-side installation package	Installation of mobile apps	Whether the installation manual is standard, concise and easy to understand.
The installation manual is complete, correct, if there are changes, the document is updated synchronously
Copy the installer directly to your computer for proper installation
Install according to the steps given in the installation manual, the installation is correct
See if the prompt information is clear during installation and whether the meaning is clear
During the installation process, click the Cancel button to exit the installer normally and the software is available.
If an SD card is identified during installation and installed to the SD card by default
During the installation process, answer the phone or SMS, the installation is successful
Whether the installer automatically checks the system for disk space
Can I abort the installation if the system is running out of disk space
After installation, the information display and file installation is correct, complete
Can you copy the appropriate files to the system folder after the software is installed?
In the software installation process, the abnormal state of sudden power failure, the program is handled properly
In the software installation process, the sudden disconnection of the abnormal state of the network, the program processing is normal
In different hardware environment, can be correct, normal, complete installation
In different network environment (2G/3G/WIFI), can be correct, normal, complete installation
Install without the required hardware configuration, correct, normal, complete installation.
In the case of the installation, all information is consistent with the last save, overwriting whether the installation can be installed again
If the installation path is not consistent with the installation, overwrite installation can be installed again
If installed, uninstall the original software, install the high version, install correctly
If installed, uninstall the original software, install the lower version, install correctly
In the case of installed, do not uninstall the original software, directly install the high version, can be installed correctly
In the case of installed, do not uninstall the original software, directly install the lower version, can be installed correctly
Can start the application normally after installation is complete
After the installation is complete, restart your phone to start the application normally
When the installation is complete, does it affect other applications
Can I add a shortcut after installation is complete?
After the installation is complete, antivirus software will treat it as a virus.
Does the shortcut point to the installation directory after installation is complete?