Android compatibility test CTS verifier-Environment Setup, test execution, results analysis

CTS Verifier is part of the CTS and needs to be done manually, primarily for testing functions that are not tested by automated test systems, such as cameras, sensors, etc. Some test items on different phones are hidden due to hardware configuration or other reasons, that is, the total number of case in CTS Verifier, depending on what features the tester supports,

The CTS verifier contains the following 15 functions in total:

· Camera

· Clock

· Device Administration

· Features

· Hardware

· JOB Scheduler

· Location

· Managed Provisioning

· Networking

· Notifications

· Others

· Projection Tests

· Security

· Sensors

· Streaming

Pre-test prep work:

1. Download the CTS Verifier test package from http://source.android.com/compatibility/downloads.html and unzip it, as shown in 1.

Figure 1 CTS verifier Test package download

2. Brush the test machine into a version that needs to be tested.

3. Install the */android-cts-verifier/ctsverifier.apk.

4. Set the language of the phone to 中文版.

5. Turn on Bluetooth, no pairing required.

6. Open and connect to available WiFi.

7. Please prepare another cell phone to test the server and client related case.

8. The data backup Tests must be tested first, because if the data backup test fails after another test, the test log will be emptied, so it must be done first.

Next we can start the test of each feature, the following is a summary of the test methods of each feature.

Camera FOV Calibration

Click on the Done button when you click on the screen after entering the test (please keep the click Frequency not too fast, no

can cause application error), continue to repeat the above operation four times, the fourth time will automatically return to the Cts-verifier main screen, and display the test results, green for pass, red for fail.

Camera Formats

Successively preview Camera0 and Camera1 in NV21 and YV12 mode, the camera preview of each resolution is correct, select Pass or fail.

Camera Intents

The test is divided into 4 main steps:

1. Click the Start Test button to start testing, press the home button to return to the desktop selection and enter the camera

After applying take a photo, return to the desktop again, click Cts-verifier, if Pass Press

Toggle click on the pass, otherwise check that the operation is correct and do this more than once

Until you can click the Pass button.

2. Click the Start Test button to start testing, press the home button to return to the desktop selection and enter the camera

After application take a video (unlimited time), return to the desktop again, click Cts-verifier,

If the pass button can be clicked, check if the operation is correct and multiple times

Do this until you can click the Pass button.

3. Click on the Start Test button to start testing, automatically jump to the camera and click Take a photo, and click Yes

To save, click the Pass button.

4. Click the Start Test button to start testing, automatically jump to the recorder and take a video,

and click Yes to save, tap the Pass button.

Camera Orientation

The test primarily tests the correctness of the camera preview already rotated view, (for example, the second of the test

, check whether the preview is correct, the preview, the photo image has been rotated 90 °, if all is correct this

Pass), as shown in 2.

Figure 2 Camera Orientation

Camera Video

The test mainly verifies the camera's shooting under different image quality. First select the CAMERA0 camera with the Low quality option and click Test. Start shooting, and later the video playback is if the check shooting is correct, then replace the high quality to continue testing, and so on.

Alarms and Timers Tests

This test mainly verifies that the alarm clock is running correctly, if all 7 tests pass, click Pass, otherwise click fail.

Show Alarms Test

Click on show Alarms, after normal to enter the alarm list, press the Back button back to the test screen, click Pass.

Set Alarms Test

Click the Set Alarm button to enter the alarm screen, after successfully adding a set of alarms, press the Back button to return to the test

Try the interface and click Pass.

Start Alarms Test

Click Set Alarm, Wait a minute, and then click Pass if the alarm goes off.

Full Alarm Test

Click the Create Alarm button to automatically enter the alarm list to see if there is a time: 01:

23, the name is "Create Alarm Test" Alarm, if any, return to click Pass (Alarm details 3)

Figure 3 Full Alarm Test

Set Timer Test

Click the Set Timer button to enter the timer interface, manually set a countdown, after the successful timing

Return to Cts-verifier and click the Pass button.

Start Timer Test

When the start Timer button is clicked, the phone automatically generates a 30s timer, and the timer rings normally

then click Pass.

Start Timer with UI Test

When you click the Start Timer button, the phone automatically enters the timer and generates a 30s timer, and then clicks Pass when the timer rings normally.

Policy Serialization Test

After entering the test, click on the "GENERATE POLICY" button, click "APPLY polcy" button again, click

"Active" button, click "OK" to activate Cts-verifier Device Manager, press the back key to return to the Cts-verifier test directory, restart the phone, re-enter the policy serialization test, if the test Xiangkanliang then click Pass, As shown in 4.

Figure 4 Policy Serialization Test

Screen Lock Test

After entering the test (the popup prompt is the description prompt, click OK), click on Force Lock

button, the phone is turned off and locked, press the power button to wake up the screen, using the current set of unlock mode to unlock, the character

If the operation is smooth, pass, otherwise fail. There is no way to unlock the screen here.

Hardware/software Feature Summary

No red fail entries have been found after entering the test, and you can click on pass,5 as shown.

Fig. 5 Hardware/software Feature Summary

GPS Test

Put the phone in an open and unobstructed environment, when all the GPS options are passed, the screen will have a pass hint. Note Sometimes the network may fail when it is unstable, and it is recommended to test several times when it fails.

NFC Test

This function is to allow the phone in contact with other hardware devices (traffic cards, access cards, etc.) Exchange data, many phones do not support this feature, please carefully confirm. If all 5 tests pass, click Pass, otherwise click fail. Currently the phone does not support, so there is no specific test steps, then update the document.

NDEF Push Sender

NDEF Push Receiver

NDEF

HCE Reader Tests

HCE Emulator Tests

USB Accessory Test

It is generally necessary to use the following ADB driver version

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "2d01", mode= "0666"

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666", add to/etc/udev/rules.d/51- Android.rules file, add the following steps:

1. Ctrl+alt+t pull up the terminal, enter "CD/ETC/UDEV/RULES.D";

2. Enter "sudo gedit 51-android.rules", then enter the user password;

Added in open 51-android.rules file: subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "2d01", mode= "0666"

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666"

3. Ctrl+s after saving, close;

However, in special cases, the phone's hardware ID will change, when the phone connects to the PC, use the command "LSUSB" to view the phone's ID, change the following command in the yellow section:

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666"

After installing the ADB driver, enter */android-cts-verifier and execute the script by command "./cts-usb-accessory". Under normal circumstances, the phone will pop up confirmation message, click Confirm, the mobile phone will send some message, the PASS option is highlighted, pass, otherwise fail,6, Figure 7 shows.

Figure 6 PC message Figure 7 Phone

Charging Constraints

Unplug the USB, then "Start Test" highlights, click "Start Test", wait a few seconds, the first case shows Green, then plug in the USB, wait a few seconds, the second case highlighted, click Pass, otherwise click fail.

Connectivity Constraints

Turn off WiFi and data traffic, "Start test" highlights, click "Start Test", wait a few seconds, the bottom three case all show green, click Pass, otherwise click fail.

Idle Mode Constraints

Turn off the screen, light up the screen again, "Start Test", click "Start Test", wait a few seconds, the bottom two case all show green, click Pass, otherwise click fail.

Battery Saving Mode Test

Click the first "LAUNCH SETTINGS", open location, go back to Cts-verifier, click the second "LAUNCH SETTINGS" and choose location mode as "Battery saving", Return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

Device only Mode Test

Click the first "LAUNCH SETTINGS" to open the location, if the previous location is open, then the first "LAUNCH SETTINGS" is grayed out. Directly click on the second "LAUNCH SETTINGS", the location mode selected as "Device only", Return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

High accuracy Mode Test

Click the first "LAUNCH SETTINGS" to open the location, if the previous location is open, then the first "LAUNCH SETTINGS" is grayed out. Directly click the second "LAUNCH SETTINGS", the location mode selected as "High accuracy", return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

Location Mode Off Test

Click "LAUNCH SETTINGS", close location, return to Cts-verifier, bottom four case all show green, click Pas, otherwise click fail.

Device Owner Provisioning

Click "Device owner Negative test", click "START PROVISIONING", Pop Up the dialog box "The Device is already set up", click OK, click Pass, otherwise click fail.

Bluetooth Test

This function is to test the phone Bluetooth, need another installed ctsverifier.apk mobile phone (called refer) with the test machine, respectively, the server and the client, testing the relevant case. If all 5 tests pass, click Pass, otherwise click fail.

Toggle Bluetooth

Click "ENABLE Bluetooth", Pop Up the dialog box, select "Allow", Bluetooth is turned on. Click "DISABLE Bluetooth", Bluetooth is turned off, pass highlighting, click Pass, otherwise fail.

Insecure Client

Refer mobile phone Open insecure Server, test machine click "SCAN for DEVICES", search to refer, click Refer, connect, click Pass, if not found, suggest the client and Server re-switch, try a few more times.

Insecure Server

Test machine Click into Insecure server,refer mobile phone Open Insecure Client, click "SCAN for DEVICES", search to test machine, click Refer, connect, then test machine Click Pass, if not found, It is recommended that client and server be re-switched and tried several times.

Secure Client

As with the insecure client operation, just enter the pairing password after searching.

Secure Server

As with the insecure server operation, you only need to enter a pairing password after searching.

Wi-Fi Direct Test

This function is to test the mobile WiFi, need another installed ctsverifier.apk mobile phone (called refer) with the test machine, respectively, the server and client or requester and responder, testing the relevant case. If all 6 tests pass, click Pass, otherwise click fail.

GO Negotiation Responder Test

The test principle is similar to Bluetooth.

GO Negotiation Requester Test

The test principle is similar to Bluetooth.

Group Owner Test

The test principle is similar to Bluetooth.

Group Client Test

The test principle is similar to Bluetooth.

Service Discovery Responder Test

The test principle is similar to Bluetooth.

Service Discovery Requerter Test

The test principle is similar to Bluetooth.

CA Cert Notification Test

This test check, when the CA certificate is installed, whether there is user notification, the inside button click, all have to notify the user's dialog box pops, that is, pass, otherwise fail.

CA Cert Notification on Boot Test

Click "Check CREDENTIALS" to view the installed certificate, then click "Install Credential", Return to Cts-verifier and click on Pass.

Notification Attention Management Test

Before testing, you need to set the following steps: Settings–>sound & notification–> interruptions–> "calls/messages from" –> "starred Contacts Only "

1. Click the first "LAUNCH SETTINGS" to open "Notification access", and if you have previously Notification access open, proceed directly to the next step.

2. Adjust the volume key, in the Volume display screen, select "None", Return to Cts-verifier, click "I ' M done", wait about 10 seconds, the bottom of a case turns green.

3. Adjust the volume button again, in the Volume display screen, select "All", Return to Cts-verifier, click "I ' M done", wait a minute or so, the bottom 9 case turns green.

3. Third adjust volume key, in the Volume display screen, select "Priority", return to Cts-verifier, click "I ' M done", wait 10 seconds or so, the bottom 2 case turns green.

Pass is illuminated, then pass, otherwise fail.

Notification Listener Test

Click the first "LAUNCH SETTINGS", open "Notification access", Return to Cts-verifier, wait about 10 seconds, the bottom five case all turn green. Click the second "LAUNCH SETTINGS", close "Notification access", Return to Cts-verifier, the bottom two case all turn green, then pass, otherwise fail.

Data Backup Test

The first time you enter this test, a dialog box will pop up (English test instructions) and click OK directly.

Click the Generate Test Data button will pop up a dialog box, click the Settings button to enter the backup, set the check back up my Data (if there is no option, please contact the open close macro), 8 is shown.

Figure 8 Data Backup

Tick and press the Back button to return to Cts-verifier, click the Generate Test Data button again to reload the

Out of the box click on the OK button and.

Run the following command (can run under XP or Ubuntu):

ADB shell Bmgr Enable true

ADB Shell Bmgr Transport

Android/com.android.internal.backup.localtransport

ADB Shell Bmgr Run

ADB uninstall Com.android.cts.verifier

ADB Install APK Path

Open Cts-verifier again enter data back Test to view the values and save the values in the picture

If consistent, click Pass, otherwise fail!

Widget Framework Test

This test examines some of the basic features of the widget framework. To test, long press anywhere on the home screen, bring up the Add Widgets feature click and add a widget named Cts-verifier to the home screen. Follow the plugin's instructions to test:

1. Click the Start Test button in the Cts-verifier widget.

2. Validates the height and width approximation of the widget.

3. Real-time verification of the height and width approximation of the widget (Long press the widget, drag and drop to change the small part

Size of the item).

4. Direct Pass

5. Direct Pass

Projection Cube Test

Click Enter, see two cubes turn, hand click, and there is a cube explosion, then pass, otherwise fail.

Projection Multitouch Test

Click to enter, a finger click on the screen, a dot appears, the other finger also click on the screen, the second dot appears, and the dot with the movement of two fingers move, pass, otherwise fail.

Projection offscreen Activity

Press the lock key to turn off the screen, after five seconds the phone will vibrate, then press the lock key to brighten the screen, do not light the screen ahead of time, otherwise the case may fail. Show "Success:xxxxx", pass key is illuminated, pass, otherwise fail.

Projection scrolling List Test

You can swipe the 50 item on the screen normally, or pass, otherwise fail.

Projection Video Playback Test

After entering the screen there is a white rectangle flashing, and accompanied by a beep, then pass, otherwise fail.

Projection Widget Test

Click the up and down keys to highlight button buttons, and manually click the button buttons, also can be highlighted, or pass, otherwise fail.

Keyguard Password Verification

Click "Set PASSWORD" to set a password that contains numbers and letters, which requires at least 9 bits in length.

Click "Change PASSWORD" to modify the password.

SUID File Scanner

Click OK to automatically scan, if the test requires that the scan is completed will show Run-as Green

Prompt, you can click the Pass button in the lower left corner to pass the test.

Accelerometer Measurement Tests

This test mainly tests the accelerator function, which is divided into six steps.

1. Place horizontally on the desktop, screen up, and click Next.

2. Place horizontally on the desktop, click Next, flip the phone quickly to make the screen face down, 5s or so after

Flip the phone to see the results (usually with a beep).

3. The right edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

4. The left edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

5. The top edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

6. The bottom edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

CTS Sensor Batching Tests

The test steps are as follows:

1. Turn on airplane mode.

2. Close "Adaptive brightness".

3. Close "auto-rotate screen".

4. Close "Stay Awake".

5. Close "Location".

6. Put the phone on the horizontal desktop, and then click "Next", then the phone screen turns black, waiting for a few minutes, and so on, the phone will beep a beep, the screen becomes bright, the test is complete. The test results are displayed at the bottom of the phone (Tests passed:xx, Tests skipped:xx, Tests failed:11), if Tests skipped:0, Tests failed:0, pass, otherwise fail.

CTS Sensor Integration Tests

Ditto.

CTS Single Integration Test

Ditto.

Gyroscope Measurement Test

This test mainly tests the gyroscope function, which is divided into six steps.

1. Place it horizontally on the desktop, screen up, and click Next.

2. Level to the desktop, click Next, quickly flip the phone to make the screen face down, 5s or so after

Flip the phone to see the results (a bunch will have a beep).

3. The right edge of the mobile phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

4. The left edge of the phone is on the horizontal level, the back of the phone is perpendicular, click Next to view the results.

5. The top edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

6. The bottom edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

Magnetic Field Measurement Tests

This case tests the magnetic field function in 8 different directions, so try to stay away from the external magnetic field. If the test fails, it is recommended to go outdoors for testing.

Sensor Batching Tests

The test steps are as follows:

1. Turn on airplane mode.

2. Close "Adaptive brightness".

3. Close "auto-rotate screen".

4. Close "Stay Awake".

5. Close "Location".

6. Click "Next" to start the test, wave in front of the screen until the test results, click "Next" to start the next test, as prompted, still in front of the screen always waved ..., please carefully review each Click Next after the operation instructions, after a few cases, should be, Walk around with your phone.

Significant Motion Tests

Click "Next", a few seconds later, the test results appear, if the background is green, then pass, red is fail, yellow is not support.

Streaming Video Quality Verifier

Because of Google issues, please connect the VPN appropriately when testing this item.

Click to play the video sequentially, if the video can play smoothly, you can click on the lower left corner of the pass button,

Through this test, if six video can pass the test, click the Pass button in the lower left corner, through the

The test.

Processing of Cts Verifier test results

After all the case, click the Save button in the upper right corner of the Cts-verifier main screen to view the test results in/sdcard/ctsverifierreports, similar to the test results file name " Ctsverifierreport-2015.03.25-17.09.01-rockchip-sofia3gr-sofia3gr-lrx22g.zip ".

Android compatibility test CTS verifier-Environment Setup, test execution, results analysis