Mobile testing Strategy

Kiran Rayachoti is a senior management program manager at Sapient Corporation. He has more than 10 years of experience in QA regulations, processes, methodologies, test automation, performance testing and commercial acceptance testing. At first he was developed, and later transferred to different areas such as pre-sales, program management and QA. He wants to define the architecture of the test system, acquire post-converged systems, manage people, improve processes and change management, project management, data-driven decisions, and develop best practices for TCOE. For key customers, Kiran successfully performed QA functions with large teams around the world, conducting QA process/automated testing for transport, e-commerce, CMS and ERP customers for end-to-end work. He was a member of a small start-up company before Tcoe, even before he created Tcoe. Kiran has recently managed the entire QA program functionality for a large millions of retail project that covers mobile web and apps. He is an active member of the QA Forum and has presented a white paper on QA strategies and automated methods, initiating seminars.

?

I have been in the IT line for more than 10 years, most of the time on quality assurance, including the test types are: manual, automation, performance, mobile testing, of course, the creation of Tcoe also cost me a lot of energy. As a result, I can see a whole bunch of different challenges presented by mobile testing, and a strategy that one must adopt to successfully implement mobile testing. Here's my full experience of testing mobile apps, including various tests with various tools.
Enterprise apps for mobile devices cover unprecedented areas of work. Many computer desktop apps are now being used on mobile devices. Mobile apps are being developed to be as sensitive and reliable as PC-side apps. To address this challenge, mobile app testing has evolved into an independent test.
In this article, I've listed the challenges that are unique to mobile business app testing, and briefly explain the key success factors, mobile tools, and methods of manual, functional, automated, and performance testing for mobile apps.

　　The development of mobile environment
It may be too much of an understatement to say that mobile apps are booming. 2010, mobile App Store records: About 6.2 billion of total sales, of which 45亿来 is downloaded from the app. By 2013, some analysts expect mobile apps to profit by more than 21 billion. 1, compared to 1.5 billion TVs, 1.4 billion credit cards, 1.3 billion landline phones, 900 million PCs and 800 million cars, mobile apps are already the world's most commonly used technology devices.

Figure 1. Mobile device ownership (compared to the number of other technology devices)

Today, there are 3.75 billion mobile device buyers worldwide, and 1.45 billion mobile device users will buy two to three mobile phones. So there's a total of 5.2 billion mobile device purchases around the world. Mobile voice calls grew by about 2% (625 billion) in 2010, while mobile data accounts grew by about 17% (295 billion).

　Mobile app Status
Mobile apps include: Mobile Web and mobile apps. Mobile web is a typical common Web page for small screen mobile devices. Its features are often more limited than mobile apps. Mobile apps are apps that are downloaded and run on mobile devices. Mobile apps have faster speeds and higher performance than mobile web, and offer a wider range of capabilities and capacities. For example, with the Amazon Mobile Web, you can shop online on your mobile device, and with Facebook, you can use your mobile device to leave a message on someone's space wall. The lower left corner of Figure 2 shows an example of mobile Web (Amazon), and the right is some examples of mobile app shortcuts (Facebook, Bank of America, Skype, etc.).

Figure 2. A mobile web and some mobile app shortcuts

The huge challenge in

Mobile testing
Unlike a PC environment, there are too many hardware and software configurations in a mobile environment that are too complex to communicate. This makes mobile Web testing and mobile app testing more complex than PC Web testing and app testing, and testing on many different devices and models is time-consuming and costly. Other challenges in the mobile test are as follows:
?? Because mobile devices and models have too many display sizes and resolutions, the image is drawn and positioned in a wide range.  
?? When an app communicates on a network boundary, the network latency is unpredictable.  
?? Real-world testing allows testers to verify what appears to be the "right" thing on the device's LCD, but hardly gets any diagnostic information that differs from the one shown on the real-computer LCD.  
?? For most performance issues (such as slow page loading), real-machine testing does not provide the specific root-cause analysis required to solve the problem  
?? With a real device, it is impossible for testers to know the URL redirection/web page internal text context.  
?? It is not feasible to seamlessly test automation on real devices.

methods and tools for moving tests
to effectively test mobile web and apps, you need emulators, emulators, and real machines. Emulators, emulators, tools, and real machines must be used together. Let's take a look at the types of mobile web and app testing that we need to focus on.
?? Usability test  
?? function test--manual  
?? User Experience Test  
?? Regression test--automatic  
?? Accessibility and security testing  
?? Performance test  
all mobile tests revolve around emulators, tools, and devices. So let's take a closer look at what emulators and simulators are!

The

Emulator (emulators)
Emulator is a hardware replication with a real mobile device. The emulator simulates the mobile device and ensures that the tester can test the mobile app on the PC without having to test on the real mobile device.
There are three types of emulators below:
?? Device emulators are typically provided by the device manufacturer. The Device emulator is for a specific device mode.
?? The operating system (OS) emulator is provided by Microsoft and Google for their respective operating systems. The OS emulator simulates a specific operating system and runs on the PC for the device. OS emulators often rely on build tools for simulating mobile environments, for example, Xcode is an iphone build tool, and Eclipse emulator is an Android build tool.
?? Browser emulators are generally available on the appropriate device website; they run on browsers, not necessarily mobile devices (that is, they can run on a PC browser). There are plenty of open-source emulators online, such as the iphone's Mobione and Android Emulator 1.5 PC. One of the big advantages of the
Simulator test is that the emulator will let you know exactly what's going on with the device's LCD, which allows the tester to debug and really open the screen to see what's going on inside, giving testers insight into the problem and making it easier for developers to fix bugs. Testers can also provide shortcuts, advanced information, and data messages to developers. This reduces the effort that some development teams spend on solving problems. The following additional benefits of using the emulator:
?? Emulators are often cost-effective because they are mostly free.
?? Because virtual devices (emulators) are under the control of their software heaps, testers can collect important information about "every element" of the content page, including internal text and direct links to Web pages.
?? Compatibility testing of multiple content, such as verifying image size or identifying broken links, can be done with precision and speed.

The

Emulator (simulators)
Device Emulator is a hardware copy of a specific device that simulates the software of the device in order to test it. Testers typically use the local browser of a PC to simulate a mobile browser. (Note: The emulator is used to test the mobile web, not the mobile app.) To get a local browser to simulate, the tester changes the user agent settings in the local browser. This method is typically used to automate functional testing. The
has an emulator that allows you to quickly and easily complete your tests without using an emulator. In addition, the simulator is very cost-effective to use them and do not buy any new software.  
simulates iphone and Android for Firefox, with a quicktest Professional tool, It is also possible to automate the implementation. But this is mainly from the function, not the appearance and the sense of perspective, and mainly for functional automation testing. To reduce manual effort/cost, a common practice is to automate functionality.  
for mobile web, this is a successful way to automate and greatly reduce effort.

test Automation
because mobile testing must be done on many different devices, browsers, and operating systems, it can be expensive and time-consuming to do all the tests manually. Test automation can reduce the time and cost associated with testing. In addition, test automation can improve the productivity of test teams. However, it is important to emphasize that automated testing is not a replacement for manual testing, it is designed to reduce the effort/time required to launch a product. Automated mobile Web testing tools are different from mobile apps. For mobile web, I used HP QTP, which facilitates the best use of functional regression testing. The QTP test tool supports test automation frameworks (keyword/data driven/hybrid). By simulating the local PC browser as a mobile browser, we can run the QTP script on the mobile web. This nicely covers the functionality of the mobile Web regression test case that must be repeated repeatedly.
for mobile app testing, test tools are selected based on the device platform. I have done pocs, but also have been mobile app automation, used QTP, Fonemonkey (open source), DeviceAnywhere and other tools. There are also mobile app testing tools on the market. But before you dive into mobile app testing, you have to set a clear goal. Because it also has its own challenges, like tool support, individual learning curve and architecture support. If the project team values the benefits of automation, they can see what tools are available for app automation. This is often useful for ecommerce/retail apps, because app stability is critical to generating revenue for the company.

　　Load and performance Testing
　　The performance of mobile web or mobile apps is the most important factor that affects the conversion rate of mobile device users. (If performance is too slow, users will leave the site.) Load and performance testing is critical to finding load and performance issues, knowing that load and performance issues are not conducive to user conversions.
For mobile web, load and performance testing can be done with HP Load Runner/performance Center. The product tests a mobile web browser by simulating a mobile browser with a local browser.
For mobile apps, it relies on the platform and architecture of the mobile app. Most mobile apps get data through the service layer. One way to perform performance testing is to load the service layer when a user accesses the app manually. For example, if the data layer is called through a Web service or rest service, the performance of these services is already being tested by the manual testers when they access the mobile app. In this way, we can get close to the actual result.

　　Test the real device using the priority platform matrix
When you test your real-world device, you must test your app on every possible device platform. For example, if you are testing an Android app, you may end up having to test on 20 or more devices (e.g. Samsung, Motorola, etc.). To avoid this situation, you can use the priority platform matrix in your tests. This method can really reduce the device you want to test. This test method consists of two steps:
1. Clear the main parameters. The main parameters are the factors that affect the importance of specific hardware and software combinations. We can prioritize according to project scope. For example, if the iphone and Android are limited, then BlackBerry and Windows will not be evaluated, and if we are building a mobile web app, we won't be able to use Windows. You need to get information from your business needs. Here are a few factors that affect the importance of this combination:
?? Manufacturers
?? Resolution
?? Aspect ratio
?? Recommendations for enterprise testing or data acquisition for a specific device or OS
2. Prepare a matrix to handle all possible combinations. To prepare a matrix that can be graded on behalf of each combination. The score is calculated by combining the relevant scores. Higher scores indicate a higher critical value for the combination.

Figure 3. Priority platform Matrix instances

　　User Experience Test (UE test)
It's wise to start the user experience test earlier in the test cycle. Most people tend to put the UE test at the end, but the UE test can reveal many issues, such as appearance, font, text color, background color, content, layout, and so on, as well as finding defects and repairing them as early as possible in the test cycle. UE testing is a style setting based on business requirements that replicates documents and content. It is important for mobile app,ue testing because even a small difference is obvious to the end user.
Therefore, the UE test must be queued to the beginning of the project, not until the end.

　Accessibility and safety testing
For accessibility testing, according to the organization Standard, the "A", "AA" and "AAA" are applicable to mobile devices (i.e. organizational standards may take precedence over any criteria, but do not include the content chosen). Some web tools that can be used for accessibility testing are wat, WAVE, Jaws, and outspoken. For mobile apps, some devices also have built-in tools available for accessibility testing. For example, the IPhone app can be tested with its embedded tools.
For security testing, the 10 owasp rules apply and require routine security checks-such as authentication checks, input validation checks, dialog management checks, cryptographic checks, app checks, and injection testing. Some of the Web tools available for security testing are: HP webinspect and Web Proxy Editor, and there are few other tools that you can use to perform security testing. For effective mobile testing, the following may be important test elements:
?? The use of emulators and simulators.
?? Test on the selected real device, at least one round of testing is about the look and feel.
?? effective implementation of test automation.
?? Thorough user experience testing: style, font, content, picture, layout.
?? performance Testing.
?? Use a variety of tools to support different tests.
?? Tested on multiple networks (depending on the range). Some companies support the service in SaaS mode, such as DeviceAnywhere.
?? Use the priority platform matrix to select the correct OS, browser, and device version so that you can cover most devices well.

　　 guidelines for mobile app testing
Be aware of the guidelines when doing mobile testing:
?? Before you begin the actual test, understand the network and the device. 
?? Simulate real-world mobile web and mobile app environments with maximum likelihood.  
?? Choose the right Automation tool.　　  
Here's what you should see in the Automation tool:  
1. It wants to support multiple platforms.  
2. It supports multiple tests.  
3. It supports end-to-end testing, even if it is necessary to connect to a third-party app.  
4. Tools are easy to learn.  
5. Test script maintenance is simple.  
?? Use the priority platform matrix on the real device to test the  
?? The UE test--and some test scripts to prioritize performance tests--should be done on the real device.  
?? An agile project's general employee ratio of 1:3 (testers: Developers) may not be suitable for mobile test projects because the device mix will be more and there are some automation challenges, so the proportion should be 1:2 (the result of the effort actually depends on the project/test scope and may not be 1:2). But for large mobile test projects, we can't manage the entire test with fewer testers without using the tools.

Copyright notice: This article from SPASVO Software Testing network: http://www.spasvo.com/news/html/20141125155150.html

Original works, reproduced when you must be in the form of hyperlinks to the original source, author information and this statement, or will be held liable.

Mobile testing Strategy