"Go" Compatibility test suite (CTS) framework user manual

Original URL: http://blog.sina.com.cn/s/blog_416166e90102v6bi.html

Compatibility test suite (CTS) framework user manual

1. Why do I need a compatibility test (hereinafter referred to as CTS)?

2, how to make my device compatible.

2.1. Match with Android compatibility definition document (ACD)

2.2, through the CTS test

2.3. Submit a report

3. How the CTS works

3.1. Work flow

3.2. Test Case Type

3.3, the current test coverage

4. Set up and use CTS

4.1. Configuring CTS

4.2. Set up your device

4.3. Using CTS

4.4. Select CTS Program

5, explain the test results

6. Release Notes

6.1. General

6.2. Known Issues

7. Appendix: CTS Console Command Reference

1. Why do I need a compatibility test (hereinafter referred to as CTS)?

1, let the app provide a better user experience. Users can choose more apps that are right for their devices. Make apps more stable.

2. Let developers design more high-quality apps.

3, through the CTS device can run Android Market.

in addition, CTS It's free, and it's simple.

2. How can I make my device compatible?

2.1. Match with Android compatibility definition document (ACD)

First look at the ACD that matches your system version, which describes the software and hardware parameters you need to have.

2.2, through the CTS test

CTS is an open-source test framework that you use to test your device for compatibility.

2.3. Submit a report

You can send the test results to [email protected]. When you submit a CTS report, you can also request access to Android Market. We are preparing special purpose testing and certification of the website and the corresponding services, we will inform you later.

3. How the CTS works

The CTS mainly consists of two components:

A test framework component that runs on a PC. Primarily used to manage the execution of test cases.

A test case that runs on a device or emulator. These use cases are written in Java apk file.

3.1. Work flow

1, the CTS can be compiled from the source code, you can also download the compiled CTS from the website

2. Install and configure CTS.

3. Connect the device to the PC.

4, run CTS. The CTS transmits the corresponding test case (that is, an apk file) to the device and runs through instrumentation, then logs the result of the run and finally deletes the test case.

Once all the test cases have been executed, you can readjust or optimize the system with reference to the test results. Then continue to run the CTS test.

5. After the test is passed, you can submit the results of CTS (the. zip file named after the result of the test) to [email protected].

3.2. Test Case Type

The CTS consists of the following three levels:

1, the unit level. Test the code unit on the Android platform. For example, a class like Java.util.HashMap.

2, functional level. A more advanced feature combined with multiple APIs.

3, program level. Execute a collection of APIs and Android runtime services by running a simple app.

Future versions will also contain the following types:

1, strength test. Test the stability of the system under high CPU operation.

2, efficiency test. For example, the number of frames rendered per second.

3.3, the current test coverage

Currently, to ensure compatibility, test cases cover these ranges

1, Signature

For each Android product, there are XML files that describe all the public APIs. The CTS contains a tool to detect whether the APIs included in these API signatures are supported in the system.

2, Platform

Test the Platform APIs described in the SDK documentation, such as the core libraries,android application framework. These APIs are required to provide:

Correct class, attribute, method signature, method behavior, error parameter handling mode

3. Dalvik VM

Tests specifically for Dalvik VMs.

4. Platform Data Model

The platform provides data to developers using ContentProvider, such as contacts,browser,settings.

5, Platform Intents

The platform provides intent for core functionality.

6, Platform permission

Some important app permissions provided by the platform

7. Platform Resources

Simple Values,drawables,nine-patch,animations,layouts,styles and themes,loading alternate resources, etc.

4. Set up and use CTS

4.1. Configuring CTS

Note: Configuring and running the CTS step has changed in version 4.0.

To run CTS, make sure that you have the latest copy of ADB installed on your machine.

To install ADB, download and install the Android SDK tool. Then add the SDK platform Tools component.

Make sure that "adb" is in your system path.

For example:

Export path=$ path:/Home/myuser/android-sdk-linux_x86/platform-tools

4.2. Set up your device

CTS can only be performed on user devices (consumer devices).

The following instructions are important, and improper configuration can result in a test timeout or a test failure:

1. The device you want to test should run a user build (Android 4.0 and later) from source.android.com

2. Please refer to http://developer.android.com/tools/device.html to set up your device.

3. Before you run CTS, make sure that your device has been burned into a user build (Android 4.0 and later).

4. The TTS (Text to Speech) file needs to be downloaded via Settings->speech synthesis->install voice data before running the CTS test. If you do not have Android Market installed, you need to install it manually.

5, ensure that the device has an SD card inserted and the card is empty. Because the CTS may modify/delete the data on the SD card.

6, on the device to do a factory reset (SETTINGS->SD Card & phone storage->factory data reset). Note: This will delete all user data on the device.

7. Make sure the device is not under any lock pattern (remove Settings->security&location->require pattern)

8. Make sure that the "USB Debug" option is selected (Settings > Developer options > USB Debugging).

9. Make sure "Stay Awake" is selected (Settings > Developer options > Stay Awake)

10. Make sure Settings > Developer options > Allow mock locations is set to True

11. Make sure the device is connected to a functioning Wi-Fi network (set >wifi network)

12,. Make sure the device displays the home screen at the beginning of the CTS (press the Home button).

13. When a device is being tested, no other tasks can be performed.

14, CTS Run, do not press any key, on the test device keys or touch the screen will interfere with the test run, and can lead to test failure.

15. Set Accessibility tests:

1. Adb Install android-cts/repository/testcases/ctsdelegatingaccessibilityservice.apk

2. On the device, enable settings > Accessibility > Delegating accessibility Service

16. Set up the device Management test:

1. Adb Install android-cts/repository/testcases/ctsdeviceadmin.apk

2. On the device, enable settings > Security > Device Administrators >

android.deviceadmin.cts.ctsdeviceadmin* settings

17. Use ADB to copy CTS media files to your device according to the instructions below:

4.3. Using CTS

Running a test plan requires:

1. Make sure you have at least one device connected to your PC. Start the CTS console by running the CTS tradefed script, for example

$./android-cts/tools/cts-tradefed

2. You can execute the default test plan by executing the Start–plan CTS. This test plan contains all the test cases.

Type list plans to view the Repository test plan (testing plans) listing, and type list packages to view the Repository test package (testing packages).

View a complete list of supported commands by referencing or typing help with the CTS command.

3. Alternatively, you can perform ctsplan on the command line using the CTS tradefed run CTS--plan

4. You should verify the progress and results reported on the console.

4.4. Select CTS Program

In the current CTS version, the following 7 test plan are included altogether.

1. CTS

Contains all the test cases, approximately 17,000 tests are performed on the device. No performance tests are included.

2, Signature

Contains signature authentication for all public APIs.

3. Android

Contains tests for the Android platform API.

4. Java

Contains tests for the Java core Library API.

5. VMs

Contains tests for Dalvik VMS.

6, Refapp

Contains tests for related applications

7, performance

Contains tests for system performance.

These can be performed by the run CTS command mentioned earlier.

5, explain the test results

Test results are placed in: $CTS _root/repository/results/.zip

In this zip file, the Testresult.xml file contains the actual test results, open this file with a browser can see the effect of the following.

The "Device Information" section provides details of the device and firmware (such as Make,model,firmware build,platform, etc.) and the hardware configuration of the device (screen parameters, keyboard, screen type, etc.).

In addition, the "Test Summary" section provides a description of the execution details of the test plan, including the CTS plan name and the start and end time of execution. There are also statistics for test results, including test pass, failure, timeout, number of failed executions.

The table below counts how many test cases in a package are passed.

This table is followed by a more specific description of the execution results.

This report lists the test package,test suite,test case and the tests performed, as well as the results of the test execution: pass, fail, timeout, not executed. When the test fails, the stack trace can be found in the XML file, and the stack trace is not included in order to make the execution result more concise. Use a text editor to view the XML file, and search for tags and tags in it.

6. Release Notes

6.1. General

Note: The CTS test in the Android4.0 version has changed a lot.

Some of the new features that have been added include splitting the CTS test to make it run on multiple parallel devices, as well as faster performance.

This CTS version contains about 17,000 tests.

Before you begin the CTS test, make sure that you set it according to 4.2 "Setting up your device". Failure to comply with these instructions may cause the test to time out or fail.

6.2. Known Issues

7. Appendix: CTS Console Command Reference

"Go" Compatibility test suite (CTS) framework user manual