App test flow and test points

1 App Testing Basic process

1.1 Flowchart

1.2 Test cycle

The test cycle can be determined by the project development cycle to determine the test time, the general test time is two or three weeks (that is, 15 working days), depending on the project situation and version quality can be properly shortened or extended test time. Confirm the project schedule with the supervisor before the formal test.

1.3 Testing Resources

Before the test task begins, examine the test resources.

--product function requirement document;

--the product prototype diagram;

--Products;

--Behavioral statistical analysis definition document;

--Test equipment (ios3.1.3-ios5.0.1;android1.6-android4.0;winphone7.1 and above; Symbian V3/v5/nokia Belle, etc.);

--Other.

1.4 daily newspaper and product launch report

1) Testers are required to send test daily to the project.

2) The contents of the test daily include:

--Grading the current test version quality;

--The more serious problems are cited, prompting the developers to change first;

--Evaluate the overall situation of the version.

3) before the product goes live, the tester sends the product on-line report.

4) The contents of the online report include:

---to rank the current version quality;

---Attach test report (Functional test report, Compatibility test report, Performance test report, and app available performance standard results);

--summarize the basic situation of the on-line version. If you have a legacy issue, you must list and record the solution.

2 app Test points

2.1 Safety Testing

2.1.1 Software permissions

1) risk of deduction: including sending text messages, making calls, connecting networks, etc.

2) Privacy disclosure risk: including access to mobile phone information, access to contact information, etc.

3) Check the application's input validity, authentication, authorization, sensitive data storage, data encryption, etc.

4) Restrict/allow access to the Internet using mobile phone function

5) Restrict/allow the use of mobile phone to send reception information function

6) Restrict/Allow application to register auto-start Application

7) Restrict or use local connections

8) Restrict/allow use of mobile phone photo or recording

9) Restrict/allow use of mobile phone to read user data

10) Restrict/allow use of mobile phone to write people user data

11) Detection of user authorization levels, data leaks, unauthorized access to the app, etc.

2.1.2 Installation and uninstallation security

1) The application should be properly installed on the device driver

2) Ability to locate the appropriate icon for the application on the installation device driver

3) whether to include digital signature information

4) all managed properties and their values contained in JAD files and jar packages must be correct

5) The data content displayed by the Jad file should be consistent with the content displayed by the application

6) The installation path should be able to specify

7) without the user's permission, the application cannot be pre-programmed to start automatically

8) Whether the uninstall is safe, and whether the files installed in it are all uninstalled

9) Whether the files generated during user use are prompted

10) Whether the modified configuration information is restored

11) Whether the uninstall affects the functionality of other software

12) Uninstall should remove all files

2.1.3 Data Security

1) When a password or other sensitive data is lost to the application, it will not be stored in the device and the password will not be decoded.

2) The password of the sender will not be displayed in clear text form

3) Passwords, credit card details, or other sensitive data will not be stored in the location of their pre-loss.

4) The individual ID or password length of the different application must be at least between 418 digit lengths

5) When the application processes credit card details, or other sensitive data, the data is not written in clear text to other separate files or temporary files. With 6) to prevent the application from terminating abnormally and without having its temporary files removed, the file may be attacked by human invaders and then read the data.

7) When sensitive data is lost to the application, it is not stored in the device.

8) Backup should be encrypted, recovery data should be considered the exception of the recovery process? Communication interruption, data recovery should be verified before use

9) The application should consider the user's prompt information or security report generated by the system or virtual machine

10) The application cannot ignore user prompts or security warnings generated by the system or virtual machine, nor can it deceive the user by displaying misleading information before the security warning is displayed, and the application should not impersonate a security warning to mislead the user

11) Before the data is deleted, the application should notify the user or the application to provide a "Cancel" command operation

12) "Cancel" command operation can be implemented according to the design requirements of its function

13) Applications should be able to handle situations where application software is not allowed to connect to personal information management

14) When you perform a read or write user information operation, the application will send the user an action error message

15) Do not damage the side except the personal information management application without the user's explicit permission μ

16) The application reads and writes the data correctly.

17) Applications should have exception protection.

18) If important data in the database is being rewritten, the user should be notified in time

19) be able to handle the errors that occur properly

20) Unexpected circumstances should prompt the user

2.1.4 Communication Security

1) In the course of running its software, if there is a call, SMS, EMS, MMS, Bluetooth, infrared and other communications or charging, whether it can suspend the program, the priority of processing communications, and after processing can be normal recovery software, continue its original function

2) When the connection is established, the application can handle the interruption of the connection due to the interruption of the network connection.

3) should be able to handle communication delay or interruption

4) The application will remain working until the communication time-out, which in turn sends the user an error message indicating a connection error

5) should be able to handle network anomalies and timely notify users of anomalies

6) Disconnect when the application shuts down or the network connection is no longer in use

7) HTTP, HTTPS overlay test

--app and background services are generally interactive through HTTP, verifying that the HTTP environment is normal;

-In the Public free network environment (such as McDonald's, Starbucks, etc.) to enter a user name and password, through SSL authentication to access the network, you need to use the HTTP client library exception to capture processing.

2.1.5 Man-Machine interface security

1) Return menu always remains available

2) command has priority order

3) Sound settings do not affect the functionality of the application

4) The application must use the full-screen size applicable to the target device to display the above content

5) The application must be able to handle unpredictable user actions, such as wrong actions and pressing multiple keys at the same time

2.2 Installing and uninstalling tests

Verify that the app is properly installed, run, uninstalled, and used for system resources before and after operation and operation

2.2.1 Installation

1) The software is installed properly under different operating systems (Palm OS, Symbian, Linux, Android, IOS, Black berryos 6.0, Windows Phone 7).

2) After the installation of the software can be normal operation, the installation of the folder and files are written to the specified directory.

3) The combination of the various options for the software installation conforms to the summary design instructions

4)) UI test of the Software Installation Wizard

5) Whether the software installation process can be canceled, after clicking Cancel, the written file is treated as a summary design note

6) Whether the process of accidental handling during software installation meets the requirements (e.g., crash, restart, power loss)

7) If the installation space is insufficient when the appropriate prompt

8) No redundant directory structures and files are generated after installation

9) for installation that requires network authentication, try it out in the case of off-screen

10) also need to test the installation manual, according to the installation manual can be installed smoothly

2.2.2 Uninstall

1) Delete the installation folder directly uninstall whether there is a prompt message.

2) test the system directly uninstall the program if there is a prompt message.

3) After testing the uninstall file, delete all the installation folders.

4) Testing of unexpected conditions during uninstallation (e.g., freezing, power-down, restart).

5) Uninstall whether to support the cancellation feature, click Cancel after the software uninstall situation.

6) The system directly uninstall the UI test, whether there is an uninstall status progress bar hint.

2.3 UI Test

Test the layout of the user interface (such as menus, dialogs, windows, and other customizable controls), whether the style meets customer requirements, whether the text is correct, whether the page is beautiful, the text, the picture combination is perfect, whether the operation is friendly, etc.

The goal of the UI test is to ensure that the user interface will provide the user with the appropriate access or functionality by testing the functionality of the object. Ensure that the user interface complies with company or industry standards. Including user-friendly, humanized, easy to operate test.

2.3.1 Navigation test

1) Buttons, dialog boxes, lists and windows, etc., or navigating between different connection pages

2) is easy to navigate, navigation is intuitive

3) Whether a search engine is required

4) navigation help is accurate and intuitive

5) The style of navigation and page structure, menu, Connection page is consistent

2.3.2 Graphics Test

1) Horizontal comparison. All controls operate in a uniform manner

2) Adaptive interface design, the content according to the window size adaptive

3) page label style is unified

4) Whether the page is beautiful

5) The picture of the page should have its actual meaning and require the overall orderly appearance

6) Picture quality is high and the picture size should be as small as possible when the design meets the requirements

7) The overall use of the interface color should not be too much

2.3.3 Content Testing

1) Input box description text content and system function is consistent

2) Whether the length of the text is limited

3) Whether the text content is unclear

4) Whether there is a typo

5) Whether the information is displayed in Chinese

6) Whether there are sensitive words, keywords

7) Whether there are sensitive images, such as: copyright, patents, privacy and other pictures

2.4 Functional Testing

According to the software description or user requirements to verify the implementation of the various features of the app, the following methods to implement and evaluate the functional testing process:

1) Use the time, place, object, behavior and background five elements or business analysis methods to analyze and refine the user use scene of the app, compare the description or demand, collate the internal, external and non-functional directly related requirements, build test points, and clear test standards, if the user needs no clear standard to follow, Refer to industry or relevant international standards or guidelines.

2) According to the characteristics of the measured function points bowl out the corresponding type of test cases to cover them, such as, the place involved in the input needs to consider equivalence, boundary, negative, abnormal or illegal, scene rollback, association test and other test types to overwrite it.

3) keep track of the coverage of the test implementation and requirements input at all stages of the test implementation, and revise the business or demand comprehension errors in a timely manner.

2.4.1 Run

1) After the app installs the test run, can open the software normally.

2) app open test, whether there is a progress prompt to load status.

3) App open speed test, whether the speed is considerable.

4) Whether the switch between the app pages is smooth and the logic is correct

5) Registration

--Same Form editing page

--User name password length

--Post-Registration Tips page

--The Front desk registration page and the Background Management page data is consistent

--after registration, in the Background management page prompt

6) Login

--Use a legitimate user login system.

--whether the system allows multiple illegal landings, and whether there are times limit.

-Use the logged-in account to log on to the system for proper handling.

--Use the Disabled account login system is correct processing.

--whether the user name, password (password) error or missing fill in the login.

-Deleted or modified users, the original user login.

--Do not enter the user password and user, repeat point (OK or Cancel button) is allowed to login.

-After landing, the landing information on the page.

--There is a logout button on the page.

--Processing of login timeouts.

7) Logout

--Write off the original module, the new module system can be handled correctly.

--Terminate the logout can return the original module, the original user.

--Write off the original user, the new user system can be handled correctly.

--Log off with an incorrect account number, password, and a disabled account with no permissions

Front and rear switch for 2.4.2 applications

1) The app switches to the background and back to the app, checking to see if it stays on the last user interface.

2) app switch to the background, then back to the app, check the function and application status is normal, IOS4 and IOS5 version of the processing mechanism are different.

3) The app switch to the background, and then back to the foreground, notice whether the program crashes, functional status is normal, especially for switching back from the background to the foreground data has automatic updates.

4) Mobile phone lock screen unlocking after entering the app note whether it will crash, functional status is normal, especially for switching back from the background to the foreground data has automatic updates.

5) When the app is in use, when the phone comes in and then switches to the app, is the function status normal?

6) When the app process is killed, the App,app can start normally.

7) The prompt box must be processed, switch to the background, and then switch back, check whether the prompt box is still there, and sometimes the application will automatically skip the hint box defects.

8) for the data exchange of the page, each page must be the front and rear switch, lock screen test, this page is most prone to crashes.

2.4.3 Free Login

Many apps offer a login-free feature that automatically uses the app as soon as the app is turned on with the user's identity.

1) When the app has a login-free feature, you need to consider the iOS version differences.

2) Consider the ability to enter the non-logon state normally without network conditions.

3) After switching user login, to verify the user login information and data content is updated to ensure that the original user exit.

4) According to the existing rules of Mtop, an account is allowed to log on only one machine. Therefore, it is necessary to check the situation that one account is logged in to multiple phones. The user in the original phone needs to be kicked out and given a friendly hint.

5) The app switches to the background, then back to the front of the check

6) switch to the background and switch back to the foreground test

7) After the password change, check if there is a valid identity verification of the data exchange

8) An application that supports automatic login checks to see if the system is automatically logged on and the data is operating correctly when the data is exchanged.

9) Check that the user is actively logged out, the next time you start the app, you should stay in the login screen

2.4.4 Data Update

According to the application business rules, as well as the amount of data update, to determine the optimal data update scheme.

1) need to determine which areas need to provide manual refresh, which need to be automatically refreshed, where manual + automatic refresh is required.

2) You need to update the data to determine where to switch back from the background to the foreground.

3) According to the reasonable allocation of business, speed and traffic, determine which content needs to be updated in real time and which needs to be updated regularly.

4) Determine the processing logic of the data Presentation section, whether it is requested from the server or cached locally, so that the corresponding test can be targeted.

5) Check where there is data exchange, there are corresponding exception handling.

2.4.5 Offline Browsing

Many apps support offline browsing, where local clients cache part of the data for users to view.

1) can browse local data without network condition

2) When you exit the app and then open the app, you can browse

3) switch to the background and cut back to the front desk for normal browsing

4) lock screen and then unlocking back to the application foreground can be viewed normally

5) When the data on the server is updated, the corresponding prompt will be given offline.

2.4.6 App Update

1) There is an update prompt when the client has a new version.

2) When the version is a non-mandatory upgrade version, users can cancel the update, the old version can be used normally. The update prompt will still appear the next time the user launches the app.

3) When the version is a forced upgrade, when the user does not make an update after the mandatory update is given, the client exits. The next time you start the app, a mandatory upgrade prompt still appears.

4) If the client has a new version, the direct update check will update properly if the client is not deleted locally.

5) When the client has a new version, if the client is not deleted locally, check whether the updated client features are new versions.

6) When the client has a new version, if the client is not deleted locally, check whether the file with the same name can be updated to the latest version properly. If the above cannot be updated successfully, also belong to the flaw.

2.4.7 positioning, Camera services

1) app useful to camera, location service, need to pay attention to system version difference

2) useful to location services, camera services, the need for a front and rear switch test, check whether the application is normal.

3) If the location service is not turned on, use the location service and the friendly popup will allow setting the location prompt. When you are sure to allow the positioning to be enabled, you can automatically jump to the location settings to turn on location services.

4) Testing the positioning, camera services, you need to use the real machine for testing.

2.4.8 Time Test

The client can set the phone's time zone and timing itself, so it needs to verify the impact of the setting on the app.

--China is the East 8 district, so when the phone set time is not East 8, see where the time needs to be displayed, whether the time is displayed correctly, the application function is normal. Time generally need to be based on the server time and then converted to the corresponding time zone of the client to display, this user experience is better. For example, to publish a microblog on the server record is 10:00, when the Washington time is 22:00, when the client to browse, if the setting is Washington time, the display of the published time is 22:00, when the time is set back to the East 8 zone time, then the view is displayed as 10:00.

2.4.9 Push Test

1) Check whether the push message is sent according to the specified business rules

2) Check that the user is no longer receiving push when the push message is not accepted.

3) If the user has set a time-out period, the user cannot receive push during the time-free period.

During a non-intrusive time period, the user receives the push normally.

4) When the push message is for the logged-on user, you need to check that the received push is consistent with the user's identity and that no error has been sent to the other person's message. In general, only the last logged-on user on the phone is sent a message push.

5) test push, you need to use a real machine to test.

2.5 Performance Testing

Evaluate the time and space characteristics of your app:

1) Limit test: In the case of various boundary pressure, such as battery, storage, speed, etc., verify that the app can respond correctly.

--Install the app when the memory is full

--Phone power off when running the app

--Disconnect the network while running the app

2) Responsiveness test: Test whether the various types of operations in the app meet user response time requirements.

--app response time for installation and uninstallation

--app the impact time of various functional operations

3) Stress test: Under the repeated/long-term operation, the system resources are not occupied abnormally.

--app load and unload repeatedly to see if the system resources are normal

--other functions repeatedly, to see if the system resources are normal

4) Performance evaluation: Evaluate the usage of system resources in typical user scenarios.

5) Benchmark Test (baseline test): Benchmarking with competitive products, product evolution comparison test, etc.

2.6 Cross-event testing

This paper presents a test method for the classification of service level and real-time characteristics of intelligent terminal applications. Cross-testing, also known as an event or conflict test, is a test in which a feature is executing while another event or operation interferes with the process. Such as: The app in front/background running state with the call, file download, music listening and other key applications such as interactive testing. Cross-event testing is important to identify potential performance issues in many applications.

1) Whether multiple apps run at the same time affect normal function

2) Does the app run pre/background switch affect normal function

3) Call/answer phone when app is running

4) Send/Receive information when the app is running

5) Send/Receive mail when app runs

6) switch Network when app is running (2G, 3G, WiFi)

7) Browse the network while the app is running

8) Use Bluetooth to transmit/receive data while the app is running

9) When the app is running, bring your own device with your camera, calculator, etc.

2.7 Compatibility test

Main test internal and external compatibility

1) compatibility with local and mainstream apps

2) based on the different development environment and production environment, the test is in various network connection (WiFi, GSM, GPRS, EDGE, WCDMA, cdma1x, CDMA2000, HSPDA, etc.), the app's data and application is correct

3) compatibility with various devices, if there is cross-system support, it is necessary to check whether the various behaviors are consistent under each system.

--compatibility with different operating systems, suitability for

--compatibility of screen resolutions of different mobile phones

--compatibility of different handset brands

2.8 Regression Test

1) After a bug is fixed and a regression test is required after the new version is released.

2) The regression test after the bug is delivered, and the regression test of the full quantity use case is carried out before delivery.

2.9 Upgrade, update test

After the release of the new version, with the different network environment of self-updating prompts and download, installation, update, launch, run verification test.

1) test whether the function after upgrade is the same as the requirement description

2) test the function of modules related to the Upgrade module to match the requirements

3) Upgrade the installation of the test (such as the crash, power, restart)

4) UI testing of the Upgrade interface

5) upgrade Test between different operating systems

2.10 User Experience Testing

To perceive the comfort, usefulness, ease of use and friendliness of a product or service in the perspective of a subjective, ordinary consumer. Through different individuals, independent space and non-empirical statistical multiplexing methods to effectively evaluate the product experience characteristics of the proposed amendments to improve the product's potential customer satisfaction.

1) Whether there is an empty data interface design, to guide users to perform operations.

2) Whether user guidance is abused.

3) Whether there is a non-clickable effect, such as: Your button is not available at this time, then must be gray, or take off the button, otherwise it will mislead users

4) is the menu level too deep?

5) Are there too many branches of the interaction process

6) Whether the relevant options are far away

7) Whether to load too much data at once

8) Whether the button in the interface can be clicked in a moderate range

9) Whether the tab is not affiliated with the content, when switching the label, the content is switched

10) The operation should have primary and secondary affiliation

11) Whether to define back logic. When dealing with hardware and software interactions, the back key should be defined specifically

12) Whether there is a horizontal screen mode design, applications generally need to support the horizontal screen mode, that is, adaptive design

2.11 Hardware Environment Testing

2.11.1 gesture Operation Test

1) The effect of mobile unlock screen on running apps

2) The impact of switching networks on running apps

3) Effect of front and rear switch in running app

4) Switching of multiple running apps

5) When the app is running, shut down

6) Restart the system when the app is running

7) Charging when the app is running

8) When the app is running, kill the process and open it.

2.11.2 network Environment

The mobile phone network is mainly divided into 2G, 3G, WiFi. At present, the 2G network phase is relatively slow, especially in testing the test of this block.

1) No network, the implementation needs the operation of the network, to give friendly tips to ensure that the program does not appear crash.

2) in the intranet test, you should pay attention to the selection of the external network operation when the exception processing.

3) When the network signal is not good, check the function status is normal, make sure not to submit data failure caused by crash.

4) When the network signal is not good, check whether the data will remain in the status of the submission, there is no timeout limit. If the data exchange fails to give a hint.

5) When the network signal is not good, after the operation, when the callback is not completed, exit this page or perform other operations, there is no abnormal situation. This issue also often occurs with program crash.

2.11.3 server outage or test in 404, 502, etc.

Background services involve DNS, space service providers, the situation will affect its stability, such as: when the domain name resolution failure, your request for the background API is likely to be 404 error, throw an exception. The exception will need to be handled correctly, or it may cause the program to not work properly.

2.12 Interface Test

The server typically provides JSON-formatted data to the client, so we need to perform interface testing on the server side to ensure that the interface provided by the server and the JSON content is translated correctly, with the corresponding return value for the branch and exception stream. This block test can be tested using the ITest framework. The most convenient is to use HttpClient for interface testing.

When conducting a service-side test, you need to develop an interface document.

2.13 Client Database Testing

1) General increase, delete, change, check test.

2) If the table does not exist, whether it can be automatically created, when the database table is deleted and can be self-built, the data can be automatically retrieved from the server and saved.

3) Whether the client can save data locally when the business needs to retrieve data from the server to the client.

4) When the business needs to fetch data from the client, can the app data be automatically removed from the client data when the client data is checked, or will it still be fetched from the server side? Check whether the app data is automatically fetched from the server side and saved to the client when the client data does not exist

5) If the business changes the data, after the deletion, the client and the server will have a corresponding update.

App test flow and test points