Threadingtest (threading test) leads white box testing into industry

threadingtest (threading test) leads white box testing into industry

The test has always been black and white. Due to the white box test generally need to have a higher technical requirements and more than the general development of the project experience and rigorous logic thinking ability, and test time is longer, more for unit testing, tools expensive, so the general domestic enterprises will ignore white box testing, which is why the white box test was born so far, There is no official promotion in the country. For a healthy test team, you must have one or more people who are familiar with white-box testing. Let us first analyze, under normal circumstances, to implement coverage testing, there are several completely different strategies.

1 black box test (functional test)

Black box testing is a function-oriented test, the test case is based on the requirements of the software, the test object is running software. Typically, a test case written by an experienced test engineer according to the requirements specification will generally reach about 70% of the coverage after execution, requiring N business days. To reach final goal 100% of coverage, new test cases need to be added as well. Friends who have experience in coverage testing know that the remaining 30% may require n 3 working days, or more, and in order to achieve higher coverage, there will often be a large number of repetitive test cases, significantly increasing the cost of testing. For black box testing coverage and cost use:

2 White Box test

In contrast to black-box testing, the test case for white-box testing is based on software code. The test engineer needs to get the test case according to the logical structure of the Code and the analysis of the basic path. Because the method is directly oriented to the code, the test case written out is very targeted, each execution of a use case, coverage indicators can have a new improvement, and ultimately achieve the ultimate goal. That sounds great, but there's a big problem, too, that testing engineers need to analyze the logical structure of all the code, call relationships, data flow, and so on, which can take a huge amount of time. In addition, the traditional unit-level white-box test focuses on program coverage, regardless of program unit combination and integrated system-level functionality. For the white box test coverage and cost of use:

Combined with the above 2 methods, Threadingtest put forward a new concept namely threading Test (theadingtest), it uses a combination of white box and black box method, with black box test method, to obtain the white box test data. Advantages of Combining:

3 Threading Test

Since the above mentioned black box and white box each have advantages and disadvantages, so it is better to use the combination of the two methods. Threading testing is actually an innovative system-level white-box test tool, which is a new academic genre in the field of software testing. It first through the traditional black box test the basic functions are tested round, coverage of 70%, while the process by the Threading Test tool monitoring, and obtain the test results data at this stage, the second step, through the threading test results of the white box, quickly locate the remaining 30% of the code, targeted to increase the test case, Ultimately achieve the ultimate goal. After many customer experience, this method is most effective for coverage testing and has the shortest test time.

Speaking of which, many testers certainly have a strong interest in threading test, then we will use the story to illustrate the traditional white-box test of the requirements of the Test Personnel division, as well as threading test in this aspect of the response strategy:

A story about code coverage

Early in the morning, a young programmer asked the master, "I'm going to write some unit test cases." How much code coverage should be achieved? ”

The master replied: "Don't consider code coverage, just write some good test cases." ”

The young programmer was delighted, bowed and departed.

Not long after, the second programmer asked the master the same question.

The master pointed to a pot of boiling water and said, "How many meters should I put into this pot?" ”

The programmer looked puzzled and replied, "How can I have an answer?" It depends on how many people you want to eat, whether they are hungry, what food you have, how many meters you have, and so on. ”

"Absolutely right," said the master.

The second programmer was delighted, bowed and departed.

Finally, the third programmer asked the master the same question about code coverage.

"80%, no less!" The master punched the hammer on the table and replied in a stern tone.

The third programmer was delighted, bowed and departed.

A story about code coverage-interpreting

After replying to this, a young intern came to the master's side:

"Master, I overheard you today." You have three different answers to the same code coverage issue. Why? ”

The master stood up from his chair: "Give me some tea, let's talk about this." ”

When the cup was filled with steaming green tea, the master began to say:

"The first programmer was a novice and was just beginning to learn the test. Currently he has a large number of programs that do not have test cases. He has a long way to go, and now it is useless to ask him for code coverage to hit him. It is better to let him slowly learn to write some test cases, test it. He can consider code coverage later. ”

"And this second programmer is very experienced in both programming and testing. I asked her how many meters she should put in the pot, so that she could understand that there are many factors that determine the number of test cases, and she knows these factors more than I do-after all, her own code. There is no simple, straightforward answer to this question. With her cleverness, she can fully understand this truth and complete the task correctly. ”

"I understand," said the young intern, "but if there is no straightforward answer, why do you tell the third programmer ' 80%, not less '? ”

The master laughed and the green tea was blown out.

"This third programmer only wants a simple answer-even if there is no simple answer ... And even if there is an answer, she will not follow the answer. ”

Young interns and grizzled masters drank tea in contemplation.

From the above story we can find:

The first novice tester needs to be trained for a long period of time for coverage testing.

The second experienced tester needs to be familiar with both programming and testing as well as the overall program being tested in coverage testing.

The third explains that the coverage metrics are clearly required by the tester when conducting coverage testing.

But in the actual operation process, the domestic due to the scarcity of white box testers, the long training period, expensive and the requirements of the Test schedule and other problems caused its slow development.

For the case of white box, threading test has to put forward a new approach.

The above mentioned the threading test through the original black box test method, to obtain the white box results, so that the test difficulty and the ability of testers greatly reduced, and on this basis, in order to make the white box test results more convenient to understand, threading test has put forward a visual code coverage, with a simple graphical display, The vast majority of black box testers who do not understand the internal structure of code and programs can also perform master-level code coverage testing.

Example: Seen as a threading test for the theory, the product of the tool Threadingtest:

Figure Coverage SC0 Explanation:

Paragraph

A sequence of computer program statements between two successive branch points is called as a segment.

"Visible Segments"

The maximum possible non-conditional statement sequence within a control layer is called a visual segment. The length of the viewable segment between two nodes may be 0 (no executable statement).

SC0

Basic segment Test coverage metrics are also known as Block Test overrides. If all visible segments (blocks) of a program are executed at least once, the SC0 coverage of that program reaches 100%.

sc0= the number of blocks executed/the number of blocks that the program contains (that is, the number of visible segments)

In the figure, we clearly see the coverage of the function SC0, how to be calculated, and show the relevant code, in this way, can make the majority of non-contact code tester, through the black box test is the way to find out the coverage of the code is not covered to the part of the test case to complement, This enhances the production of test cases and improves test quality.

In Threadingtest, there is also a description of other coverage, such as true (percent of true conditions), BOTH (percent of coverage for true and false), Branch (branch coverage), MC/DC, and so on.

Please pay attention to the coverage analysis in the official technical website http://www.teststars.cc/, with detailed explanations and calculations.

Test coverage effect

Test coverage is part of the test end criteria

Test coverage for modules with low coverage and critical modules. This data can help us quickly locate the modules that need more testing, and can help us understand the testing situation of the important modules to measure the quality of our test cases and even the quality of our tests.

In a spiral development model, if we don't control the test coverage in our last iteration, when one version accumulates, it's hard to determine which modules are not adequately tested during development.

Develop effective test plans for the next phase through coverage

Report for test coverage

Through the coverage show, we can carry out the overall direction plan for the next test.

Check for unused features

Check the minimum coverage for the first 10

Strengthening of test cases

Threading test coverage and validation phase

The validation phase can be divided into the unit validation (UT) phase, the integrated validation (IT) phase, and the system validation (ST) phase.

The unit validation phase is concerned with module functionality and module quality, at which point the export condition is code coverage. General industry commonly used export conditions are: line coverage reached 100%, branch coverage reached 100%, the condition coverage reached 95%, no coverage of the need to give a reasonable explanation.

The integration verification phase, the function of the concerned system, and the interface between the module and the module, at this time the export condition is the function coverage. General industry commonly used export conditions are: the functional coverage of 90%, the need to give a reasonable description of no coverage.

• High functional coverage and low code coverage:

Verify that the plan is insufficient and that you need to add functional coverage points.

• High code coverage and low functional coverage:

The design does not implement the specified functionality.

Threading to meet test coverage for best practices

Traditional white-Box testing

Path coverage > Condition overrides > Decision overrides > Statement overrides

Test coverage 100% is an ideal situation that is difficult to achieve

Test coverage 100% does not indicate that we did a complete test.

The test coverage is achieved by taking into account the overall coverage of the software, as well as the cost of the project, including manpower, time and so on.

Because of the above factors, so the traditional white box test does not recommend the company deliberately to meet the coverage test indicators, for testing and testing.

Threading test for the traditional white-box test results of the unified management of test data, to achieve the accumulation of various stages, shorten the time of repeated testing, so as to ensure the test 100% coverage of high quality.

From the original test, the unit test stage, the integration test stage and the test data of the system test phase are separated from each other, but in the actual process, the degree of the unit test is related to the integration test, the system test and so on. In this part of the threading test using the cumulative coverage of the technology, the entire test phase of the test results are inherited and accumulated, so that the entire test iteration has been the role of quantitative correlation, can be at any stage of the test analysis and improvement.

Compared to the traditional unit-level white box test, the threading test also puts forward the distribution test method, for medium-sized software or Web site, a single tester is not able to complete the entire test task, in order to better cooperate with each other, Threadingtest uses the distributed test design, in the test process, Testers can test the different modules of a program or website at different locations, and the test results are aggregated to the central server without interfering with each other, so that each person's test data results are managed in a unified way, thus effectively quantifying the overall test progress.

The appearance of threading test to the reform of testing field

Commercial test Tool products threadingtest The threading concept of the actual product, through the tool, so that the black box testers can also carry out code-level white box testing, and the entire test phase of the process of quantitative management, through the black box test to achieve white box results display, The test method is completed by combining the most effective 70% black box +30% white box in the Test field.

Threading test broke the traditional white box test operation is difficult, too much pursuit of coverage, and so on, through the combination of black box and white box, so that all stages of testing staff, can correctly according to their own needs to test, so as to avoid blindness, repetition, omission and other issues.

Threadingtest (threading test) leads white box testing into industry