"Go" Android Compatibility Test CTS verifier-Environment Building, test execution, results analysis

Original URL: http://www.cnblogs.com/zh-ya-jing/p/4452675.html

CTS Verifier is part of the CTS and needs to be done manually, primarily for testing functions that are not tested by automated test systems, such as cameras, sensors, etc. Some test items on different phones are hidden due to hardware configuration or other reasons, that is, the total number of case in CTS Verifier, depending on what features the tester supports,

The CTS verifier contains the following 15 functions in total:

· Camera

· Clock

· Device Administration

· Features

· Hardware

· JOB Scheduler

· Location

· Managed Provisioning

· Networking

· Notifications

· Others

· Projection Tests

· Security

· Sensors

· Streaming

Pre-test prep work:

1. Download the CTS Verifier test package from http://source.android.com/compatibility/downloads.html and unzip it, as shown in 1.

Figure 1 CTS verifier Test package download

2. Brush the test machine into a version that needs to be tested.

3. Install the */android-cts-verifier/ctsverifier.apk.

4. Set the language of the phone to 中文版.

5. Turn on Bluetooth, no pairing required.

6. Open and connect to available WiFi.

7. Please prepare another cell phone to test the server and client related case.

8. The data backup Tests must be tested first, because if the data backup test fails after another test, the test log will be emptied, so it must be done first.

Next we can start the test of each feature, the following is a summary of the test methods of each feature.

Camera FOV Calibration

After entering the test click on the screen to take a photo after clicking the Done button (please keep the click Frequency not too fast, otherwise it may cause application error), continue to repeat the above operation four times, the fourth time will automatically return to the Cts-verifier main interface, and display the test results, green for pass, red for fail.

Camera Formats

Successively preview Camera0 and Camera1 in NV21 and YV12 mode, the camera preview of each resolution is correct, select Pass or fail.

Camera Intents

The test is divided into 4 main steps:

1. Click the Start Test button to start testing, press the home button to return to the desktop selection and go to the camera app to take a photo, return to the desktop again, click Cts-verifier, if the Pass button can click the Pass, otherwise please detect the operation is correct and many times, Until you can click the Pass button.

2. Click the Start Test button to start testing, press the home button to return to the desktop selection and enter the camera app to take a video (unlimited time), return to the desktop again, click Cts-verifier,

If the pass button clicks the Pass, then check that the operation is correct and do this multiple times until you can click the Pass button.

3. Click on the Start Test button to start testing, automatically jump to the camera and click Take photos, and click Yes to save the pass button.

4. Click the Start Test button to start testing, automatically jump to the recorder after shooting a video, and click Yes to save, click the Pass button.

Camera Orientation

The test mainly tests the correctness of the camera preview has been rotated, (for example, the second item of the test, whether the preview is correct, whether the image is rotated 90 °, if both are correct), 2 is shown.

Figure 2 Camera Orientation

Camera Video

The test mainly verifies the camera's shooting under different image quality. First select the CAMERA0 camera with the Low quality option and click Test. Start shooting, and later the video playback is if the check shooting is correct, then replace the high quality to continue testing, and so on.

Alarms and Timers Test

This test primarily verifies that the alarm clock is running correctly, and if all 7 tests pass, click Pass, otherwise click fail.

Show Alarms test

Click Show Alarms, after normal access to the alarm list, press the Back button back to the test screen, click Pass.

Set Alarms test

Click on the Set Alarm button to enter the alarm screen, after successfully adding a set of alarms, press the back button back to the test screen, click Pass.

Start Alarms Test

Click Set Alarm, Wait a minute, and then click Pass if the alarm goes off.

Full Alarm Test

Click the Create Alarm button to automatically enter the alarm list to see if there is a time: 01:23, the name "Create Alarm Test" Alarm, if any, return to click Pass (Alarm details Information 3)

       

                                Figure 3 Full Alarm Test

Set timer Test

Click the Set Timer button to enter the timer interface, manually set a countdown, after the successful timing returned CTS-VERIF IER, click the Pass button.

Start timer Test

When you click the Start Timer button, the phone automatically generates a 30s timer, and the timer starts to sound and then clicks Pass.

Start timer with UI Test

When the Start Timer button is clicked, the phone automatically enters the timer and generates a 30s timer, and then clicks Pass when the timer rings normally.

Policy Serialization Test

After entering the test, click the "GENERATE POLICY" button, click the "APPLY polcy" button again, click the "ACTIVE" button, click "OK" to activate Cts-verifier Device Manager, press the Back button to return to the Cts-verifier test directory, Restart your phone, enter policy serialization test again, and if you test Xiangkanliang, click pass,4 as shown.

Figure 4 Policy Serialization Test

Screen Lock Test

After entering the test (the pop-up prompt is a description prompt, click OK), click on the Force Lock button, the mobile phone out of the screen and locked, press the power button to wake the screen, using the current settings to unlock the way to unlock, in line with the smooth operation of the pass, otherwise fail. There is no way to unlock the screen here.

Hardware/software Feature Summary

No red fail entries have been found after entering the test, and you can click on pass,5 as shown.

Fig. 5 Hardware/software Feature Summary

GPS Test

Put the phone in an open and unobstructed environment, when all the GPS options are passed, the screen will have a pass hint. Note Sometimes the network may fail when it is unstable, and it is recommended to test several times when it fails.

NFC Test

This function is to allow the phone in contact with other hardware devices (traffic cards, access cards, etc.) Exchange data, many phones do not support this feature, please carefully confirm. If all 5 tests pass, click Pass, otherwise click fail. Currently the phone does not support, so there is no specific test steps, then update the document.

NDEF Push Sender

NDEF Push Receiver

NDEF

HCE Reader Tests

HCE Emulator Tests

USB Accessory Test

It is generally necessary to use the following ADB driver version

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "2d01", mode= "0666"

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666", add to/etc/udev/rules.d/51- Android.rules file, add the following steps:

1. Ctrl+alt+t pull up the terminal, enter "CD/ETC/UDEV/RULES.D";

2. Enter "sudo gedit 51-android.rules", then enter the user password; in the open 51-android.rules file, add:

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "2d01", mode= "0666"

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666"

3. Ctrl+s after saving, close;

However, in special cases, the phone's hardware ID will change, when the phone connects to the PC, use the command "LSUSB" to view the phone's ID, change the following command in the yellow section:

subsystem= "USB", attrs{idvendor}== "18d1", attrs{idproduct}== "e781", mode= "0666"

After installing the ADB driver, enter */android-cts-verifier and execute the script by command "./cts-usb-accessory". Under normal circumstances, the phone will pop up confirmation message, click Confirm, the mobile phone will send some message, the PASS option is highlighted, pass, otherwise fail,6, Figure 7 shows.

Figure 6 PC message Figure 7 Phone

Charging Constraints

Unplug the USB, then "Start Test" highlights, click "Start Test", wait a few seconds, the first case shows Green, then plug in the USB, wait a few seconds, the second case highlighted, click Pass, otherwise click fail.

Connectivity Constraints

Turn off WiFi and data traffic, "Start test" highlights, click "Start Test", wait a few seconds, the bottom three case all show green, click Pass, otherwise click fail.

Idle Mode Constraints

Turn off the screen, light up the screen again, "Start Test", click "Start Test", wait a few seconds, the bottom two case all show green, click Pass, otherwise click fail.

Battery Saving Mode Test

Click the first "LAUNCH SETTINGS", open location, go back to Cts-verifier, click the second "LAUNCH SETTINGS" and choose location mode as "Battery saving", Return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

Device only Mode Test

Click the first "LAUNCH SETTINGS" to open the location, if the previous location is open, then the first "LAUNCH SETTINGS" is grayed out. Directly click on the second "LAUNCH SETTINGS", the location mode selected as "Device only", Return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

High accuracy Mode Test

Click the first "LAUNCH SETTINGS" to open the location, if the previous location is open, then the first "LAUNCH SETTINGS" is grayed out. Directly click the second "LAUNCH SETTINGS", the location mode selected as "High accuracy", return to Cts-verifier, the bottom four case all show green, click Pas, otherwise click fail.

Location Mode Off Test

Click "LAUNCH SETTINGS", close location, return to Cts-verifier, bottom four case all show green, click Pas, otherwise click fail.

Device Owner Provisioning

Click "Device owner Negative test", click "START PROVISIONING", Pop Up the dialog box "The Device is already set up", click OK, click Pass, otherwise click fail.

Bluetooth Test

This function is to test the phone Bluetooth, need another installed ctsverifier.apk mobile phone (called refer) with the test machine, respectively, the server and the client, testing the relevant case. If all 5 tests pass, click Pass, otherwise click fail.

Toggle Bluetooth

Click "ENABLE Bluetooth", Pop Up the dialog box, select "Allow", Bluetooth is turned on. Click "DISABLE Bluetooth", Bluetooth is turned off, pass highlighting, click Pass, otherwise fail.

Insecure Client

Refer mobile phone Open insecure Server, test machine click "SCAN for DEVICES", search to refer, click Refer, connect, click Pass, if not found, suggest the client and Server re-switch, try a few more times.

Insecure Server

Test machine Click into Insecure server,refer mobile phone Open Insecure Client, click "SCAN for DEVICES", search to test machine, click Refer, connect, then test machine Click Pass, if not found, It is recommended that client and server be re-switched and tried several times.

Secure Client

As with the insecure client operation, just enter the pairing password after searching.

Secure Server

As with the insecure server operation, you only need to enter a pairing password after searching.

Wi-Fi Direct Test

This function is to test the mobile WiFi, need another installed ctsverifier.apk mobile phone (called refer) with the test machine, respectively, the server and client or requester and responder, testing the relevant case. If all 6 tests pass, click Pass, otherwise click fail.

GO Negotiation Responder Test

The test principle is similar to Bluetooth.

GO Negotiation Requester Test

The test principle is similar to Bluetooth.

Group Owner Test

The test principle is similar to Bluetooth.

Group Client Test

The test principle is similar to Bluetooth.

Service Discovery Responder Test

The test principle is similar to Bluetooth.

Service Discovery Requerter Test

The test principle is similar to Bluetooth.

CA Cert Notification Test

This test check, when the CA certificate is installed, whether there is user notification, the inside button click, all have to notify the user's dialog box pops, that is, pass, otherwise fail.

CA Cert Notification on Boot Test

Click "Check CREDENTIALS" to view the installed certificate, then click "Install Credential", Return to Cts-verifier and click on Pass.

Notification Attention Management Test

Before testing, you need to set the following steps: Settings–>sound & notification–> interruptions–> "calls/messages from" –> "starred Contacts Only "

1. Click the first "LAUNCH SETTINGS" to open "Notification access", and if you have previously Notification access open, proceed directly to the next step.

2. Adjust the volume key, in the Volume display screen, select "None", Return to Cts-verifier, click "I ' M done", wait about 10 seconds, the bottom of a case turns green.

3. Adjust the volume button again, in the Volume display screen, select "All", Return to Cts-verifier, click "I ' M done", wait a minute or so, the bottom 9 case turns green.

3. Third adjust the volume key, in the volume display interface, select "Priority", return to Cts-verifier, click "I ' M done", wait 10 seconds or so, the bottom 2 case turn green, pass bright, then pass, otherwise fail.

Notification Listener Test

Click the first "LAUNCH SETTINGS", open "Notification access", Return to Cts-verifier, wait about 10 seconds, the bottom five case all turn green. Click the second "LAUNCH SETTINGS", close "Notification access", Return to Cts-verifier, the bottom two case all turn green, then pass, otherwise fail.

Data Backup Test

The first time you enter this test, a dialog box will pop up (English test instructions) and click OK directly.

Click the Generate Test Data button will pop up a dialog box, click the Settings button to enter the backup, set the check back up my Data (if there is no option, please contact the open close macro), 8 is shown.

Figure 8 Data Backup

Tick and press the Back button to return to Cts-verifier, click the Generate Test Data button again, click on the OK button in the popup box.

Run the following command (can run under XP or Ubuntu):

ADB shell Bmgr Enable true

ADB Shell Bmgr Transport

Android/com.android.internal.backup.localtransport

ADB Shell Bmgr Run

ADB uninstall Com.android.cts.verifier

ADB Install APK Path

Open Cts-verifier again to data back Test to see whether the values are consistent with the values in the saved image, or if they are consistent, click Pass, otherwise fail!

Widget Framework Test

This test examines some of the basic features of the widget framework. To test, long press anywhere on the home screen, bring up the Add Widgets feature click and add a widget named Cts-verifier to the home screen. Follow the plugin's instructions to test:

1. Click the Start Test button in the Cts-verifier widget.

2. Validates the height and width approximation of the widget.

3. Real-time verification of the height and width approximation of the widget (after long pressing the widget, drag and drop to change the widget size).

4. Direct Pass

5. Direct Pass

Projection Cube Test

Click Enter, see two cubes turn, hand click, and there is a cube explosion, then pass, otherwise fail.

Projection Multitouch Test

Click to enter, a finger click on the screen, a dot appears, the other finger also click on the screen, the second dot appears, and the dot with the movement of two fingers move, pass, otherwise fail.

Projection offscreen Activity

Press the lock key to turn off the screen, after five seconds the phone will vibrate, then press the lock key to brighten the screen, do not light the screen ahead of time, otherwise the case may fail. Show "Success:xxxxx", pass key is illuminated, pass, otherwise fail.

Projection scrolling List Test

You can swipe the 50 item on the screen normally, or pass, otherwise fail.

Projection Video Playback Test

After entering the screen there is a white rectangle flashing, and accompanied by a beep, then pass, otherwise fail.

Projection Widget Test

Click the up and down keys to highlight button buttons, and manually click the button buttons, also can be highlighted, or pass, otherwise fail.

Keyguard Password Verification

Click "Set PASSWORD" to set a password that contains numbers and letters, which requires at least 9 bits in length.

Click "Change PASSWORD" to modify the password.

SUID File Scanner

Click OK to automatically scan, if the test is completed after the scan will show Run-as green hints, you can click on the lower left corner of the pass button to pass the test.

Accelerometer Measurement Tests

This test mainly tests the accelerator function, which is divided into six steps.

1. Place horizontally on the desktop, screen up, and click Next.

2. Horizontally placed on the desktop, click Next, quickly flip the phone to make the screen face down, 5s and then flip the phone to view the results (usually there will be a beep).

3. The right edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

4. The left edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

5. The top edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

6. The bottom edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next. View the results.

CTS Sensor Batching Tests

The test steps are as follows:

1. Turn on airplane mode.

2. Close "Adaptive brightness".

3. Close "auto-rotate screen".

4. Close "Stay Awake".

5. Close "Location".

6. Put the phone on the horizontal desktop, and then click "Next", then the phone screen turns black, waiting for a few minutes, and so on, the phone will beep a beep, the screen becomes bright, the test is complete. The test results are displayed at the bottom of the phone (Tests passed:xx, Tests skipped:xx, Tests failed:11), if Tests skipped:0, Tests failed:0, pass, otherwise fail.

CTS Sensor Integration Tests

Ditto.

CTS Single Integration Test

Ditto.

Gyroscope Measurement Test

This test mainly tests the gyroscope function, which is divided into six steps.

1. Place it horizontally on the desktop, screen up, and click Next.

2. Level to the desktop, click Next, quickly flip the phone to make the screen face down, 5s and then flip the phone to view the results (a bunch will have a beep).

3. The right edge of the mobile phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

4. The left edge of the phone is on the horizontal level, the back of the phone is perpendicular, click Next to view the results.

5. The top edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

6. The bottom edge of the phone is placed on the horizontal plane, the back of the phone is perpendicular, click Next to view the results.

Magnetic Field Measurement Tests

This case tests the magnetic field function in 8 different directions, so try to stay away from the external magnetic field. If the test fails, it is recommended to go outdoors for testing.

Sensor Batching Tests

The test steps are as follows:

1. Turn on airplane mode.

2. Close "Adaptive brightness".

3. Close "auto-rotate screen".

4. Close "Stay Awake".

5. Close "Location".

6. Click "Next" to start the test, wave in front of the screen until the test results, click "Next" to start the next test, as prompted, still in front of the screen always waved ..., please carefully review each Click Next after the operation instructions, after a few cases, should be, Walk around with your phone.

Significant Motion Tests

Click "Next", a few seconds later, the test results appear, if the background is green, then pass, red is fail, yellow is not support.

Streaming Video Quality Verifier

Because of Google issues, please connect the VPN appropriately when testing this item.

Click to play the video sequentially, if the video can play smoothly, you can click on the lower left corner of the pass button,

Through this test, if six video can pass the test, click the Pass button in the lower left corner to pass the test.

Processing of Cts Verifier test results

After all the case, click the Save button in the upper right corner of the Cts-verifier main screen to view the test results in/sdcard/ctsverifierreports, similar to the test results file name " Ctsverifierreport-2015.03.25-17.09.01-rockchip-sofia3gr-sofia3gr-lrx22g.zip ".

You can see the Android-cts-verifier-4.4_r3 test method Summary:

Http://wenku.baidu.com/link?url=MVuUeCknWWjhcxmKC827gVVV7oYgPcJ36eGdf9NJyn4Lb_F-rgXjk0YfydSyEjy1eXPIoWj2e_ Rq50u8hpkqnaik5hoqasecrjaoc1jh_l3

"Go" Android Compatibility Test CTS verifier-Environment Building, test execution, results analysis