Android compatibility testing framework (CTS) Manual

This document is translated by Android official CTS manual android-cts-manual-r4.pdf

 

Android compatibility testing framework Manual

 

 

1. Why does the compatibility test (CTS) be required )?

1.1. Make the app provide a better user experience. You can select more apps suitable for your devices. Make apps more stable.

1.2. developers can design high-quality apps.

1.3. devices using CTS can run Android
Market.

In addition, CTS is free and simple.

 

2. How can I make my device compatible.

2.1. With Android
Compatibility definition document (hereinafter referred to as ACD) Matching

Take a look at the ACD that matches your system version. This document describes the required software and hardware parameters.

2.2 pass CTS Test

CTS is an open-source testing framework used to test whether your device is compatible.

2.3 report submission

You can send the test results to the cts@android.com. When you submit a CTs report, you can also request access to Android
Market.

We are preparing websites and corresponding services for special purpose testing and certification. We will notify you after the preparation.

 

3. How CTS works

CTS mainly contains two components:

Testing Framework components running on the PC. It is mainly used to manage test cases (Test
Case.

Test cases running on devices or simulators. These use cases are APK Files written in Java.

 

3.1 Workflow

1. You can compile CTS from the source code or download the compiled CTS from the website.

2. install and configure CTS.

3. Connect the device to the PC.

4. Run CTS. CTS transmits the corresponding test case (that is, an APK file) to the device and runs it through instrumentation. Then it records the running results and deletes the test case.

After all the test cases are executed, you can refer to the test results to re-adjust or optimize the system. Run the CTS test again.

After the 5th test passes through, you can submit the result of the ctsgenerated (.zip file under the pull resultnamed by test time) to the cts@android.com.

 

1.  

   1. Test Case Type

CTS has three levels:

1. unit level. Test the code unit on the Android platform. For example, a java. util. hashmap class.

2. functional level. A more advanced function combined by multiple APIs.

3. program level. Run a simple app to execute an API set and Android runtime service.

Future versions will also include the following types:

1. strength test. The stability of the test system under high CPU operations.

2. Efficiency Test. For example, the number of frames rendered per second.

 

3.3. Current Test Coverage

Currently, to ensure compatibility, the test cases cover the following scopes:

1. Signature

Each Android product has some XML files to describe all public APIs. CTS contains a tool to check whether all the APIs in these API signatures are supported in the system.

2. Platform

Test the platform API described in the SDK documentation, such as core
Libraries, Android Application Framework, etc. These APIs are required to provide:

Correct class, attribute, method signature, method behavior, and error parameter Handling Method

3. Dalvik
VM

For Dalvik
Test the VM.

4. Platform
Data Model

The platform provides data to developers through contentprovider, such as contacts, browser, and settings.

5. Platform
Intents

Intent provided by the platform for core functions.

6. Platform
Permission

Important app permissions provided by the Platform

7. Platform
Resources

Simple
Values, drawables, nine-patch, animations, layouts, styles and
Themes, loading alternate resources, etc.

 

4. Configure and use cts

4.1 configure cts

You can run CTS only in version 1.6 or later.

Decompress the zip package, edit the Android-CTS/tools/startcts script, and modify the sdk_root variable to match the environment.

For example:

Sdk_root =/home/myuser/android-sdk-linux_x86-1.6_r1

 

That is, point to the SDK root directory.

4.2 configure the device

The following descriptions are important. Improper configuration may result in test timeout or test failure:

1. Download the SDK to the machine.

2. the device you want to test should run a user
Build.

3. Refer to this link (http://developer.android.com/guide/developing/device.html) to set up your device.

4. Before Running CTS, make sure that your device has a user
Build

5. Run the settings-> speech command before running the CTS test.
Synthesis-> install voice data to download the TTS file. If Android is not installed
You need to install it manually.

6. We recommend that you use a Google account dedicated for testing to log on to the device.

7. Make sure that the device has an SD card and the SD card is empty. Because CTS may modify/Delete the data on the SD card.

8. Perform a factory reset (settings-> SD) on the device.
Card & phone storage-> factory data reset ). Note: This will delete all user data on the device.

9. Make sure the device is not in any lock
Under pattern (cancel settings-> Security & location-> require
Pattern)

10. Ensure that "screen"
Timeout is set to "Never
Timeout "(settings-> sound & Display-> Screen
Timeout should be set to "Never
Timeout ")

11. Ensure "Stay"
Awake is selected (settings-> Applications-> development-> Stay
Awake)

12. Make sure that settings-> application-> development-> allow
Mock locations is set to true.

13. When running CTS, the device stays on the desktop.

14. When the settings are being tested, no other tasks can be executed.

15. Do not press any key or touch the screen when CTS is running.

 

4.3 Use cts

Run a test
Plan needs:

1. At least one device or simulator is connected to the PC. Then run the script Android-CTS/tools/startcts.

2. You can run start
-Plan CTS to run the default test
Plan. This test plan contains all test cases.

Use ls
-P can view the Repository

List of test cases contained in.

Use ls
-Plan to view the Repository

Included Test
Plan list.

3. You can also run startcts.
Start-plan <plan_name> to run a specified test
Plan.

 

4.4 select cts
Plan

The current CTS version contains the following seven
Plan.

1. CTS

Contains all test cases. About 121000 tests are executed on the device.

2. Signature

Includes signature authentication for all public APIs.

3. Android

Including testing the Android platform API.

4. Java

Includes testing of Java core library APIs.

5. VM

Includes
Test the VM.

6. refapp

Includes tests on related applications

7. Performance

Includes testing of system performance.

 

5. Explain the test results.

Test results are stored in $ cts_root/Repository/results/<start
Time>. Zip

. In this ZIP file, the testresult. xml file contains the real test results. Open this file in a browser and you can see the following results:

"Device"
Information Section provides device and firmware details (such as make, model, firmware
Build, platform, etc.) and hardware configurations of the device (screen parameters, keyboard, screen type, etc ).

In addition, "Test
The Summary section provides the test
Detailed description of plan execution, including cts
Plan Name and execution start and end time. The test results include the number of tests that pass, fail, time-out, and cannot be executed.

The following table lists how many tests are passed in a package.

This table is followed by a more detailed description of the execution results.

This report lists Test
Package, test suite, test
Case and executed tests, as well as test execution results: Pass, fail, time-out, not executed. When the test fails, you can find the stack in the XML file.
Trace, to make the execution results more concise, these stacks
Trace is not included. Use a text editor to view the XML file and search for the <Test> tag and <stacktrace> tag.

 

6. Notes

CTS restarts the device during testing. This is normal.

CTS can only be executed on one device at a time.

When running CTS, the force close dialog box may pop up, asking users to choose to close or wait. Re-running this test will usually be fine.

 

 

There are several articles below, which are concise. You can read these articles first and then read this article, which makes it easier to understand.

Http://chenhuawei1234567.blog.163.com/blog/static/194526712010629111638224/

Http://blog.csdn.net/zjujoe/archive/2010/06/01/5640461.aspx