Threadingtest extended method to break the results and display the new concept of testing

Current mobile testing formats

As the platform with the largest number of apps today, mobile businesses have penetrated into everyone's life, which means that the testing requirements for mobile platforms are also increasing year by year, major Testing Tool developers have also released their own testing tools and platforms based on market needs.

The mobile black box method makes testers suffer

As the last barrier of the entire program, the tester's testing method plays a decisive role. On a mobile platform, testers usually click on a real machine. Although testin, devstore, Android SDK emulator, and other cloud platforms or automatic tools help, however, testers are still looking forward to a simple, intuitive, and easy-to-use tool to quantify their workload.

Domestic testing tools and cloud platforms

Observing mobile testing tools and platforms in China today is derived from the original black box testing tools. Compared with traditional white box testing tools, it also intuitively reflects software problems. However, its disadvantages are also obvious. For example, it is difficult to identify some special types of errors. When the software is modified, it must be re-tested, and blindness is serious.

The world's first commercial mobile white box testing tool for pure software

Thradingtest product brochure content outline

Threadingtest

Threadingtest ("TT" for short) is a white box testing tool developed by Shanghai zero one assembly Co., Ltd. Its design is based on innovative testing philosophy-"threading testing ". TT, through a series of patented invention technologies and product design, perfectly integrates the black box testing and white box testing processes and methods for the first time, and generates white box testing data using the black box testing process and methods. At the same time, TT is also the world's first commercial mobile white box testing tool for pure software applications. It can be used to test various types of mobile applications. TT is an intelligent testing tool for in-depth quantitative analysis of software testing processes and results.

"Threading test" concept \\

The name of the threading test is a pictogram name, indicating that it is used to concatenate the outputs in each stage of development and testing. The threading test is similar to the devops concept in the industry, it establishes the connection between development and testing and breaks the information gap between the two. Through a set of automatic and non-managed technical means, it achieves the close relationship between development and testing and QA, this greatly improves the quality of testing, the maintenance efficiency of large-scale software, and reduces costs.

Through automated plug-in technology, data transmission technology, and timestamp technology, the threading test is fully automated in the process of establishing association between massive data, comparison between multiple versions, merging of coverage rates, and testing, automatically establish the association between source code and test cases to achieve cabling throughout the test process.

The threading test is a new contribution to the testing theory following the black box testing and white box testing theories, and the threadingtest (personal free), a commercial product, is released along with the theory.

Fifth-generation white box Testing Technology

As a new generation of white-box testing technology, threadingtest integrates many new functions compared with previous White-box testing technologies:

1. Accumulative coverage technology: if there are multiple tested program version coverage results, TT can merge the coverage rates of multiple versions and display them in one view.

2. Coverage rate collection without supervision: the coverage rate statistics can be fully collected by the background program and transparent to testers.

3. coverage rate results and program structure display: TT supports 9 coverage rates, 6 medium complexity results visualization, and supports displaying coverage rates on function call charts, control flowcharts, and other graphics, testing and developers can clearly view the program coverage from multiple perspectives.

4. distributed test: the coverage rate generated when multiple testers access the system. The coverage rate can be displayed in a unified view.

5. Implementation of DO-178B MC/DC white box structure testing technology: To achieve 100% coverage rate, visual complex conditions combination, greatly improve product quality.

6. coverage Rate Calculation visualization: TT supports visualization of various covered computing and provides visual and intuitive results for each coverage rate calculation result, the coverage visualization interface displays the covered data and expected statistical data of modules in the unit of functions in a graphical manner. The user can intuitively obtain the calculation value of coverage through the statistical data on the interface.

1. guide the Team to improve the test cases: through the fifth-generation white-box testing technology, we can see the coverage of any sub-condition of the compound condition of each judgment statement in the control flowchart, the entire development and testing team can see the contribution of the coverage of each case to the overall test in real time. Dynamically adjusts the entire test process based on the growth of coverage rate and other indicators. It can also guide supplementary use cases for key modules with low cumulative coverage rate.

Visual Testing

Threadingtest analyzes code-level compilation and runs the program after the plug-in, TT provides a wide range of visual interfaces, including the function call diagram, control flowchart, coverage visualization view, accumulative coverage view, multi-version comparison view, and version test data report view. You can use each view to mine targeted information to quickly locate problems.

Function call diagram:

A series of overall information about the software system can be presented to users. For example, the total number of member functions of a class or function, the depth, hierarchy, total number of statements, and overall complexity of the call or class inheritance relationship, overall Test Coverage Rate (based on the cumulative results and the last running results, you can select the statement, branch, and MC/DC test coverage rate Standard) the overall performance analysis results, the proportion of time used by each module, and the analysis results of global and static variables are also provided. The specific information of each module is also given, including: the source code lines and complexity of each module, test coverage analysis results, fan-in and fan-out information, highlighting a module and all its related modules, or generating a local subtree with any module as the root.

Function call diagram features:

1. Supports ultra-high-speed graphics rendering technology with millions of elements.

2. Supports full screen, ultra-smooth zoom-in and zoom-out, and smooth mobile technology.

3. You can group and aggregate graphs by class and define the number of layers according to the function call relationship.

4. Supports step-by-step subtree expansion and drill-down.

5. display of coverage and function-related information on function elements.

6. traceable: The function call diagram is "active", not static. You can select any module in the diagram and trace all the paths and related modules that call it and the modules that are called by him, used to prevent module inconsistency defects.

7. Interactive operations can be performed. For example, a module and its related modules can be highlighted, or the call relationship of a module can be drawn separately.

8. You can select any module in the function call diagram to generate a local subgraph for the root, and generate relevant information of the subgraph, so that each group or individual of the team can conveniently obtain relevant local information.

9. link each module to the logical block diagram: complete the combination of a macro (function call diagram) and a micro view (control flowchart) to achieve better visualization.

Function call aggregation diagram:

Group functions by class and display functions of the same class in charts.

Class inheritance diagram:

Displays the integration and derivation relationships of all classes in the current project.

Control Flowchart

Use the IF-Else, while, for, do-while, switch-case and other control statement structures of the function to draw a diagram of the Process relationship. Use the following source code display interface to show the process relationship, display clearly shows the internal running logic and structure of the function, the true and false running status of the condition, and the success rate of MC/DC.

Features:

1. Visual control flow chart. The main control logic statements should be clearly displayed on elements, supporting nested display and serial display.

2. Click each element in the control flow chart to view the corresponding code segment, the number of executions and coverage of the code segment.

3. In the code segment of the control flow chart, when the condition statement becomes the selected hotspot, you can see the true and false combinations of the sub-conditions of the Condition Statement.

4. Support for thumbnail display, smooth scaling, and full screen display.

5. It can be automatically linked, traced, and converted with the function call diagram.

Function list:

All functions of the entire program are sorted by various coverage and complexity, helping you quickly locate and view all function information, the quick switchover with function call diagram, control flowchart, coverage visualization diagram, and accumulative coverage diagram makes it easy for users to view and solve practical problems.

Visualization of coverage calculation process:

Threadingtest uses the latest coverage algorithm to visualize the code algorithm, so that the tester can clearly see the code algorithm corresponding to the coverage value of each function block during the test, avoid Blind tests and repeated tests to quickly complete test cases with learning.

Bidirectional automatic association and tracing of test cases and codes

Use threadingtest to run the test case and use ultra-high-speed graph rendering technology at the TT millions of elements level to perform functional logic Implementation Analysis for various large, medium, and small software, tracing between test cases and tested source code. This tracing technology allows you to conveniently view and design test cases. By analyzing test cases based on bidirectional tracing, testers can quickly track and modify the scope of the Code, the cost of regression testing can be minimized for the modified and affected modules.

Forward tracing:

Click a test case, trace back to the function control diagram corresponding to the test case, and display the test logic and structure. Click a function, you can view the coverage, complexity, code, control flowchart, and other information of the function to help testers identify test omissions through a simple view, and help developers directly locate the defects found by the test, efficient interaction between testing and development. Imagine that when a core engineer leaves, he will take away his understanding of the entire program and his development ideas. The transferred engineer will spend a lot of time to understand it. tt uses forward tracing, this allows engineers to quickly master and understand program development ideas through the information such as program logic and structure displayed in test cases.

Reverse tracing:

Click a function in the function list to trace the function call diagram, control flow chart, and source code corresponding to the function. Click the source code, back to the list of test cases executed by the source code. Imagine that when a program requires frequent changes and upgrades, the stress on testers is enormous. Testers cannot accurately determine whether the code changed by developers affects other functions, the program can only be tested again. However, through reverse tracing, the tester clearly sees the test case table affected by the changed code, and through the cumulative coverage technology, this greatly shortens the regression testing time.

※Shanghai zero one assembly Information Technology Co., Ltd. has a patent for "a test device and method based on case and source code bidirectional traceability ".

Real-time tested application oscilloscope

Threadingtest is the first to introduce the concept of a test oscilloscope. During the actual test, you can view the speed and frequency of various logic execution in the program in real time, testers can switch from a traditional black box test of the tested application (only functional feedback can be seen, but internal feedback within the program cannot be seen) to an oscilloscope similar to a hardware test, real-time analysis and viewing of key test data throughout the test process.

Functions of the oscilloscope:

1. Record the running information of the tested program in real time: blocks, functions, conditions, and running stacks.

2. Record the program execution logic corresponding to the test case in Real Time:

3. Test Mode supporting source code separation

4. supports distributed real-time recording of running information of tested programs

5. Visually observe the program running status. Waveforms intuitively reflect program behavior.

6. Running Performance on various hardware platforms through waveform analysis programs.

7. support access to devices such as real machines and Simulators

8. Records of test data generated by various automated operations and unit tests

9. Supports connections from USB, WiFi, Bluetooth, and other devices.

Code Separation Test

The technical features of threadingtest are very suitable for testing mobile smart products, such as smart driving systems. With its own patented technology, TT can use the patented technology of the TT oscilloscope. Testers can even operate the SMART driving system while traveling, use the TT software oscilloscope to instantly obtain test data and test results. Testers can perform tests on a mobile vehicle and then take the test data obtained throughout the day to the software platform of the lab or R & D center for analysis. At the same time, TT supports offline testing and is the best way to protect the source code. That is, when testing outdoors, the source code of the system is not required, but the data presentation is required, you only need to return the data obtained from the test to the R & D lab for associated presentation and analysis.

※Shanghai zero one assembly Information Technology Co., Ltd. owns the patent "an oscilloscope and method used for Software Testing Data Analysis ".

Version comparison and cumulative coverage

During the software development process, multiple versions are inevitably generated due to multiple iterations and incremental development, and multiple version tests are also generated accordingly. Threadingtest provides the function-level comparison and test coverage merging functions for multiple versions.

Function-level comparison

Multi-version comparison, in the unit of functions, lists the comparison and evolution of a series of states such as adding, modifying, and deleting all functions (it can also be said that it is a version tracing mechanism ), combined with the function call diagram and control flowchart, you can quickly identify the differences between functions of different versions and the impact of changes to a module on other modules, and improve the ability to prevent defects.

Test engineers can modify test cases in different versions through multi-version comparison to find hidden problems caused by the upgraded version.

Sustainable Testing

Another major feature of Multi-version comparison is coverage rate merging. for multiple versions, intelligent coverage rate merging is implemented. The previous multi-version comparison at the function level is based on the coverage rate merging algorithm, accurately calculates the cumulative coverage of the latest version, which greatly reduces repeated testing tasks by test engineers.

Zero-cost integration unit testing, automated testing, and other third-party tools.

Threadingtest Android junittest is a unit test component based on androidjunit test. It utilizes the advantages of threadingtest. On the basis of the original Android JUnit test driver code, you can modify it to a unit test that complies with the threadingtest Android JUnit test code by performing a simple global replacement of characters. In addition, there is almost no difference between the two tests. You only need to enable TT Real-time Monitoring to receive data during the test, but no other operations are required during the test, you cannot even feel the data collection process.

For traditional unit tests, after a unit test is completed, the tester can only obtain whether the test results meet the expected results. If there is a difference with the expected results, the cause of the error cannot be obtained immediately. When threadingtest Android JUnit test is used for the unit test, the original test experience is still available, but the data results of the white box test can be obtained. After the unit is executed, you can analyze program error points and missing test points based on the generated data to make bug search faster and more accurate.

For example, a integrated unit test is displayed. threadingtest performs automated code insertion without changing the original test method, the TT oscilloscope receives and records the data during the test. After the test is completed, it uses the test case script to automatically generate test cases and related test data.

Open APIs

Threading also opens up API interfaces for integration, so that the original automated operations do not need to be changed, connect through the tt api interface, and the TT oscilloscope obtains the test data of other testing tools, the relationship between the test case and code and function coverage will be automatically generated for the corresponding test case. Currently, open Java APIs directly use Java as a unit test tool for script prediction. Automated testing tools can be integrated with TT through this interface. In this way, you can record the code execution logic information of the test case in TT while running the test case.

Demonstration: test scenario description:

When other test tools are used for automated testing, if there is an automated testing process that cannot be intervened or is in progress during the testing process, you need to differentiate the test cases during the testing process, for example, if you log on and exit from an automated test and execute the two test cases at the same time, you can add the start and end points of the two test cases in the program, to distinguish two test cases. In this way, we can use TT to automatically establish the relationship between the test case and the source code. For example, in this example, we will establish the code corresponding to the login operation and the code corresponding to the exit action.

Test procedure:

Take logon and exit as an example. If the following code structure exists:

... Classhomepage { Public void login (){ ... // Logon related operations } Public void logout (){ ... // Exit related operations } }

1. Introduce the JavaParser-Android.jar package into the project;

2. importcom. ZOA. writetomq. unittest;

3. Add the start and end identification.

Add the Start mark of the logon test case at the beginning of the login function, and add the end mark of the logon test case before the function ends. Similarly, you can also add the corresponding identifier at the beginning and end of the logout function. The added code is:

... Import com. ZOA. writetomq. unittest; Classhomepage { Public void login (){ Unittesttestlogin = new unittest ("homepage/login "); ... // Logon related operations Testlogin. Close (); } Public void logout (){ Unittesttestlogout = new unittest ("homepage/logout "); ... // Exit related operations Testlogout. Close (); } }

4. use TT to insert a program and perform the original test on the inserted app. During the test, TT receives and records the data through the oscilloscope, generate Test Cases and corresponding test data.

Support integration testing of distributed applications

When your environment is a distributed system, that is, when multiple components provide external services in the integrated environment, TT supports joint testing of various components in the distributed environment and their interactions. That is, you can see all the execution of an external request between multiple subsystems.

TT allows multiple users to test the tested applications at the same time. Different users are not confused with the test data generated by testing at the same time for different cases.

Reports

Threadingtest provides a user-friendly graphical report interface to fully demonstrate complexity and coverage.

Including basic information about projects, coverage and complexity statistics, coverage growth statistics, and ranking of 7 statistical information, including almost all important information. It is displayed in a bar chart, line chart, and list to quickly understand the structure and test coverage of the project.

Implemented agile testing tools

Agile Development emphasizes adaptability rather than preference, and emphasizes human factors first. Tt also provides effective support for this development method, which is quite different from the traditional development model.

1. like agile development, agile testing has higher requirements for testers. Testers need to have a deep understanding of the Code logic. TT can visually display the logical diagram of each test case, combined with the control flow chart and function call diagram, You can accurately and deeply grasp the logic structure of the source code.

2. Fast iteration is one of the features of agile development. tt uses bidirectional tracing to link source code, visual logical block diagram, and test cases to support full information sharing between them.

3. rapid iteration will inevitably produce multiple versions. tt provides intelligent Evolutionary Comparison of multiple versions and merge the coverage rate of multiple versions based on version recognition.

4. Combined with Bidirectional tracing and multi-version Evolutionary Comparison, we can minimize the scope of regression testing and select the least test cases for testing purposes.

Application Value of threading Testing

Reduce the costs of software developers, demand sides, and third-party Testers by at least 25%. Breaks barriers to communication between software developers and testers and establishes a transparent tracing mechanism.

1. Establish efficient interaction and connection between development and testing through technical means to greatly improve work efficiency and software quality. It completely changes the boring and non-technical status of the testing process, and the testing process becomes interesting and can be monitored in real time.

2. The fifth generation of coverage technology will officially bring white box testing technology into the industry.

3. Quantify the software testing system and establish perfect testing quality audit standards.

4. Works with agile development to support incremental development, demand changes, and Defect Prevention.

5. The simple and concise operation mode is simple and easy to understand. It can be promoted to developers and testers around the world.

6. Taking the lead in implementing professional testing tools for commercial use in the context of accelerated IOE for security events.

For example, IOV is an industrial application of threading testing.

As the world's first mobile white box testing tool, threadingtest can provide aerospace-Level Function Testing for any Android-based smart Driving System Based on the iso26262 software testing standard, according to the information currently retrieved, threadingtest is the only commercial tool in the world that can meet this standard.

The threadingtest kernel contains a large number of complex operations, but the front-end is presented to the user, but it is very easy to use. Like X-ray, threadingtest can thoroughly analyze every detail of the software. It is a good helper for software development engineers, test engineers, and software enterprise managers of on-board systems. For the iso26262 standard, threadingtest can perfectly cover the software testing options in the iso2626262 standard, and support and exceed the full-process testing standard in iso2626262, from unit testing to integration testing, system testing, and UAT, a unified quantitative system can be provided.

Supported Product Versions

Applicable to multiple languages: Java, objective-C, C/C ++, etc.

Multiple platforms: Android, windows, Linux, UNIX, Mac

Threadingtest extended method to break the results and display the new concept of testing