Android CTS test and androidcts Test

Android CTS test and androidcts Test

1. What is CTS?

CTS Introduction: Compatibility Test suite series Compatibility Test
Google defines a compatibility specification (Compatible Definition) to ensure that a test complies with this specification. Therefore, Android-based applications can run on devices based on the same api version. Devices tested by CTS can obtain the android trademark and enjoy the permission of the android market.

For example, mobile phones, tablets, and on-board entertainment systems. Strictly speaking, Android systems that download third-party applications must pass the CTS test to achieve compatibility.

The CTS test package can be found at: release (about CTS test). The CTS test package is downloaded (the test package will be updated continuously), for example, android-cts-verifier-4.0.3_r1-linux_x86-arm.zip,android-cts-media; decompress the downloaded CTS test package to the corresponding folder. The android-cts folder contains three folders: Docs, Repository, and Tools. You need to configure the environment variables related to the SDK, preferably in Ubuntu, and set up the Android development environment. For more information about how to configure environment variables, see.

Install apk: adb install android-cts/repository/testcases/CtsDeviceAdmin.apk before testing.

Generated in system Settings> Security> Device Manager.

Ii. Run CTS

aaa@aaa-All-Series:~/D/android-cts-4.4_r3-linux_x86-arm/android-cts/tools$ lscts-tradefed  cts-tradefed.jar  ddmlib-prebuilt.jar  hosttestlib.jar  junit.jar  README  tradefed-prebuilt.jar

First Run./cts-tradefed in the android-cts-4.4_r3-linux_x86-arm/android-cts/tools directory
If you do not have permission to run the command, run chmod 777 cts-tradefed.

aaa@aaa-All-Series:~/D/android-cts-4.4_r3-linux_x86-arm/android-cts/tools$ ./cts-tradefed

In this case
Cts-tf>

cts-tf > run cts --help'cts' configuration: Runs a CTS plan from a pre-existing CTS installationPrinting help for only the important options. To see help for all options, use the --help-all flag  cmd_options options:    --[no-]help          display the help text for the most important/critical options. Default: false.    --[no-]help-all      display the full help text for all options. Default: false.    --[no-]dry-run       build but don't actually run the command.  Intended as a quick check to ensure that a command is runnable. Default: false.    --[no-]noisy-dry-run build but don't actually run the command.  This version prints the command to the console.  Intended for cmdfile debugging. Default: false.    --[no-]loop          keep running continuously. Default: false.  test options:    --plan               the test plan to run.    -p, --package        the test packages(s) to run.    -c, --class          run a specific test class.    -m, --method         run a specific test method, from given --class.    --continue-session   continue a previous test session.  'file' logger options:    -l, --log-level-display                         the minimum log level to display on stdout. Default: ERROR. Valid values: [VERBOSE, DEBUG, INFO, WARN, ERROR, ASSERT]

Eg:
Execute a class
Cts-tf> run cts -- class libcore.javax.net. ssl. SSLSocketTest
Execute methods in a class
Cts-tf> run cts-c libcore.javax.net. ssl. SSLSocketTest-m test_SSLSocket_getSupportedCipherSuites_names
Test Results

cts-tf > run cts -c libcore.javax.net.ssl.SSLSocketTest -m test_SSLSocket_getSupportedCipherSuites_names12-02 20:01:51 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB12-02 20:01:51 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.01.5112-02 20:02:05 I/FIH6WK5PFAOFO7MB: Collecting device info12-02 20:02:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:02:07 I/FIH6WK5PFAOFO7MB: Test package android.core.tests.libcore.package.libcore started12-02 20:02:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:02:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_names FAIL junit.framework.AssertionFailedError: Unknown cipher suites expected:<[]> but was:<[TLS_FALLBACK_SCSV]>at libcore.java.security.StandardNames.assertValidCipherSuites(StandardNames.java:842)at libcore.java.security.StandardNames.assertSupportedCipherSuites(StandardNames.java:852)at libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_names(SSLSocketTest.java:68)at java.lang.reflect.Method.invokeNative(Native Method)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)12-02 20:02:18 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_5252779086477145420.zip12-02 20:02:19 I/FIH6WK5PFAOFO7MB: Saved log host_log_8790846039917450950.zip12-02 20:02:19 I/FIH6WK5PFAOFO7MB: android.core.tests.libcore.package.libcore package complete: Passed 0, Failed 1, Not Executed 012-02 20:02:19 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.01.51/testResult.xml12-02 20:02:19 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.01.51. Passed 0, Failed 1, Not Executed 012-02 20:02:19 I/FIH6WK5PFAOFO7MB: Time: 0scts-tf > cts-tf > cts-tf > cts-tf > cts-tf > cts-tf > cts-tf > cts-tf > run cts --class android.widget.cts.AbsListViewTest12-02 20:02:32 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB12-02 20:02:32 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.02.3212-02 20:02:42 I/FIH6WK5PFAOFO7MB: Collecting device info12-02 20:02:44 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:02:44 I/FIH6WK5PFAOFO7MB: Test package android.widget started12-02 20:02:44 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:02:47 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessCacheColorHint PASS 12-02 20:02:47 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessFastScrollEnabled PASS 12-02 20:02:48 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessListPadding PASS 12-02 20:02:49 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessScrollingCacheEnabled PASS 12-02 20:02:49 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSelectedItem PASS 12-02 20:02:50 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSelector PASS 12-02 20:02:50 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessSmoothScrollbarEnabled PASS 12-02 20:02:51 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessStackFromBottom PASS 12-02 20:02:51 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAccessTranscriptMode PASS 12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testAddTouchables PASS 12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testBeforeAndAfterTextChanged PASS 12-02 20:02:52 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testCheckLayoutParams PASS 12-02 20:02:53 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testComputeVerticalScrollValues PASS 12-02 20:02:53 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testConstructor PASS 12-02 20:02:54 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testDraw PASS 12-02 20:02:54 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testFoo PASS 12-02 20:02:55 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGenerateLayoutParams PASS 12-02 20:02:56 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetContextMenuInfo PASS 12-02 20:02:56 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetFocusedRect PASS 12-02 20:02:57 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testGetTopBottomFadingEdgeStrength PASS 12-02 20:02:57 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testHandleDataChanged PASS 12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testInvalidateViews PASS 12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testLayoutChildren PASS 12-02 20:02:58 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testPointToPosition PASS 12-02 20:02:59 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetFilterText PASS 12-02 20:03:03 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetOnScrollListener PASS 12-02 20:03:03 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetRecyclerListener PASS 12-02 20:03:04 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testSetScrollIndicators PASS 12-02 20:03:04 I/FIH6WK5PFAOFO7MB: android.widget.cts.AbsListViewTest#testShowContextMenuForChild PASS 12-02 20:03:09 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_5736317118032050398.zip12-02 20:03:11 I/FIH6WK5PFAOFO7MB: Saved log host_log_5498353359267291055.zip12-02 20:03:11 I/FIH6WK5PFAOFO7MB: android.widget package complete: Passed 29, Failed 0, Not Executed 012-02 20:03:11 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.02.32/testResult.xml12-02 20:03:11 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.02.32. Passed 29, Failed 0, Not Executed 012-02 20:03:11 I/FIH6WK5PFAOFO7MB: Time: 0s

cts-tf > run cts --class libcore.javax.net.ssl.SSLSocketTest12-02 20:34:51 I/TestInvocation: Starting invocation for 'cts' on build '4.4_r3' on device FIH6WK5PFAOFO7MB12-02 20:34:51 I/FIH6WK5PFAOFO7MB: Created result dir 2014.12.02_20.34.5112-02 20:35:06 I/FIH6WK5PFAOFO7MB: Collecting device info12-02 20:35:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:35:07 I/FIH6WK5PFAOFO7MB: Test package android.core.tests.libcore.package.libcore started12-02 20:35:07 I/FIH6WK5PFAOFO7MB: -----------------------------------------12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_HandshakeCompletedListener PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_HandshakeCompletedListener_RuntimeException PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_TrustManagerRuntimeException PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_clientAuth PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_clientAuth_bogusAlias PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_close PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_confirmSessionReuse PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnableSessionCreation PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnabledCipherSuites PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getEnabledProtocols PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSSLParameters PASS 12-02 20:35:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSession PASS 12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_connect FAIL java.lang.Exception: One or more problems in test_SSLSocket_getSupportedCipherSuites_connect:Problem trying to connect cipher suite TLS_FALLBACK_SCSV client=AndroidOpenSSL server=AndroidOpenSSLat libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_connect(SSLSocketTest.java:114)at java.lang.reflect.Method.invokeNative(Native Method)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedCipherSuites_names FAIL junit.framework.AssertionFailedError: Unknown cipher suites expected:<[]> but was:<[TLS_FALLBACK_SCSV]>at libcore.java.security.StandardNames.assertValidCipherSuites(StandardNames.java:842)at libcore.java.security.StandardNames.assertSupportedCipherSuites(StandardNames.java:852)at libcore.javax.net.ssl.SSLSocketTest.test_SSLSocket_getSupportedCipherSuites_names(SSLSocketTest.java:68)at java.lang.reflect.Method.invokeNative(Native Method)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1710)12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getSupportedProtocols PASS 12-02 20:36:15 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_getUseClientMode PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_interrupt PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_interrupt_read PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_reusedNpnSocket PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnableSessionCreation_client PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnableSessionCreation_server PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnabledCipherSuites PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setEnabledProtocols PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSSLParameters PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoTimeout_basic PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoTimeout_wrapper PASS 12-02 20:36:18 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setSoWriteTimeout PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setUseClientMode PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_setUseClientMode_afterHandshake PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_shutdownCloseOnClosedPeer PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake_noClientCertificate PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_startHandshake_noKeyStore PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_SSLSocket_untrustedServer PASS 12-02 20:36:20 I/FIH6WK5PFAOFO7MB: libcore.javax.net.ssl.SSLSocketTest#test_TestSSLSocketPair_create PASS 12-02 20:36:24 I/FIH6WK5PFAOFO7MB: Saved log device_logcat_9056351731602008844.zip12-02 20:36:25 I/FIH6WK5PFAOFO7MB: Saved log host_log_1029983955603589903.zip12-02 20:36:25 I/FIH6WK5PFAOFO7MB: android.core.tests.libcore.package.libcore package complete: Passed 33, Failed 2, Not Executed 012-02 20:36:25 I/FIH6WK5PFAOFO7MB: Created xml report file at file:///home/dzt/D/dzt/xxxxxxxxx/xuna/android-cts-4.4_r3-linux_x86-arm/android-cts/tools/./../../android-cts/repository/results/2014.12.02_20.34.51/testResult.xml12-02 20:36:25 I/FIH6WK5PFAOFO7MB: XML test result file generated at 2014.12.02_20.34.51. Passed 33, Failed 2, Not Executed 012-02 20:36:25 I/FIH6WK5PFAOFO7MB: Time: 0s

Test_SSLSocket_getSupportedCipherSuites_connect

Test_SSLSocket_getSupportedCipherSuites_names

These two failures are Google's problems in Android 4.4, so ignore them.