How Enterprises Implement white box testing

In the it r & D field, individual capabilities have a great impact on R & D efficiency. Developers with strong personal abilities usually have two to three levels of productivity ~ 5 times. In certain circumstances, the gap is even increased to 10 times. Therefore, in software R & D projects, we heard the Project Manager complain that several technical experts can determine the success or failure of a large product. Why is it so difficult to successfully evolve an individual to a successful team?

White-box testing of embedded products is one of the most difficult software processes to be promoted by it R & D. Personal Qualities Have a particularly significant impact on the testing results. We all know that testing a white box is based on testing tools, but some excellent employees only rely on the tools.CodeInsert assert and print statements in, and also do a good job in white box testing. This article attempts to analyze the main factors for successful white box testing, and how to standardize these factors and promote them to the entire organization, guides individuals to succeed in the team.

1. Efficient testing is the basic prerequisite for promoting white box testing

The bottleneck of white-box testing is that the testing efficiency is far lower than the R & D progress.

Excellent employees can perform tests without testing tools. First, they are used to ensuring that the test process is efficient. For example, programmers always clarify their ideas before coding, coding and writing use cases are all one-time jobs, while common developers want to write and adjust while writing, and there will inevitably be a lot of repetition; for example, experts always do as little debugging as possible, try to replace debugging with testing. when problems arise, they can be solved only by looking at the source code. If the developer who is just getting started does not follow the debugger step by step, he will lose the courage to continue his life; furthermore, developers always like to write code for a test. In a stable system at any time, the efficiency of problem detection and problem solving is very high, ordinary people prefer to write the code completely and then perform tests in a centralized manner. All kinds of problems are entangled, so it is not easy to locate and solve.

Testing without the help of testing tools may also be done well, but it does not mean that good tools are not used. There is a misunderstanding here. Generally, we think that writing a script is a formal test. Actually, it is not. We embed an Assert command in the tested code. It can be executed repeatedly and the result can be automatically judged, in essence, this is a test case. In addition to poor maintainability, this form of use case is highly efficient in development and can directly reuse the running process of the tested code, using scripts to construct the test environment and initiate a test is omitted. Therefore, "efficient testing" rather than "using tools" is the basic prerequisite for promoting white box testing.

There is an indicator-test year-on-year comparison-that can be used to assess whether the white-box test of embedded products is effective. The test year-on-year comparison is to ensure constant software quality, the development investment required for each additional 1 thousand lines of code, compared with the new test design investment, the former development investment mainly refers to: Low-layer design, coding and debugging, and the latter testing design investment mainly refers: write and debug use cases.

Based on our empirical data, the test year-on-year ratio is 0.5, that is, the code written for one day needs to be tested for two days, indicating that the current test efficiency is very low. At this time, the implementation of the white box test is hard to escape the failure. The test year-on-year ratio is 1, that is, the code written for one day must be tested for one day, which indicates that the test efficiency is acceptable. In this case, if the white box test has a strong execution intensity, such as well-defined process support, the software process can still be successful if the QA audit is complete, but it is easy to fail if the execution is insufficient. When the test year-on-year ratio reaches 2, that is, the test can be complete after one day of code writing, and the white-box test enters a virtuous circle. With a little guidance, we can change from passive test to active test.

For example, the test year-on-year curve is divided into three intervals: Forced push zone, organizational push zone, and spontaneous push zone. When the test year-on-year comparison is too low, only the cost-free mandatory promotion can make the white box test succeed, and the test year-on-year increase to more than 2, means that a little investment is a huge benefit, R & D personnel continue to test, fewer bugs are exposed, and the efficiency is improved without having to work overtime every day. White-box testing is easy to solidify into their spontaneous habits.

In practice, we often use the aforementioned "partition PUSH theory" to evaluate the white-box testing capability of an organization. The results are very consistent with the actual situation. The test year-on-year ratio is the inflection point, and the test efficiency is near the inflection point. The white box test can be cautious and optimistic, but if it is far lower than the inflection point, the situation will be bad. We have met many products. Although the product group tests are very important, the methodology and support tools they follow are not efficient, or the organization is not well-developed, the white box test is doomed to fail before it starts.

Since testing efficiency is so important, how can we improve it? The 4th white-box testing method overview has been thoroughly discussed. We can consider the testing methods and support tools. Below we will list several key points:

1. The test logic can be described in a formal and scripted manner to improve the test efficiency.

2. Online Testing can greatly improve the efficiency of case design, debugging and maintenance.

3. the debugging operation is reused to the test design, which also greatly improves the test efficiency.

4. The continuous integration mode enables code stability to be reused in debugging and testing, which effectively improves the overall R & D efficiency.

2. Test Based on interface specifications

The reason why white box testing is hard to do is that the testing results vary from person to person. It is easy to find deep-seated problems with strong abilities or a weak sense of responsibility. The white box testing is just a pass, and the effect is not obvious. At present, regardless of the sense of responsibility, we mainly discuss the technical level. How can we ensure that white box testing is in-depth and effective?

For a good employee, his design case is always used to backtrack the specification, because before coding, he has already figured out the specification, whether or not there is a document, it is not difficult for backtracing and Inspection specifications. Of course, the specifications are large and small, and they are hierarchical. at the product level, there are product features and specifications. At the code level, there are different levels of interfaces, such as unit testing, the design case is based on the input and output of the tested function.

For employees with poor work habits, code is written and thought-provoking. The software specifications in his mind are vague, random, and constantly changed. In this case, it is difficult for him to design the use case based on the specifications. The results are not hard to imagine. On the one hand, the code is written by him, and he will fall into his own habits of thinking. On the other hand, he only pays attention to the currently visible code, invisible code (such as the default else branch and missing processing process) is ignored. We call this test a "mechanical test". In other words, testers treat themselves as artifacts, for example, we can see that the Code has a sentence "1 + 1 = 2". Therefore, when design case Test 1 is added with 1 equal to 2, the test must pass, and the coverage rate can also be up to standard, but the test is still not complete, the tester is only responsible for the compiler or lint tool.

In an organization, excellent employees and ordinary employees always exist in a certain proportion. It is difficult for us to ask ordinary employees to become excellent immediately after simple training, to extend individual success to organizational success, we should first consider improving engineering methods. TDD and test-driven development in extreme programming can solve the above problems well. TDD requires testing and design before coding. The code is not visible yet. First, there are test cases, in this way, we will not fall into the trap of "mechanical testing" in any case. According to the statistics provided by the literature, TDD practices are more effective in general, but there are also some inherent defects. For example, it is easy to make people feel at a loss when they do not see the code. The 4th-generation white box testing method has been optimized and improved in practice, making the testing first (TDF, test design firt in the 4th-generation white box method) easy to operate.

Iii. smooth and continuous promotion of white box testing based on organizational Behaviors

If the white box test is successfully extended from an individual to an organizational unit, it also emphasizes that the software process can continue to operate and run smoothly, Which is standardized and reproducible.

First of all, we need to solve the test and evaluation problem. It is well-known to evaluate the degree of coverage. But is it enough to evaluate the coverage of code alone? The answer is no. Among the commercial white box testing tools, the most common coverage standards include statement coverage, branch coverage, and mcdc coverage. These coverage indicators all reflect the degree of coverage in the actual code, but do not accurately reflect the implementation degree of the test intent. That is to say, there is a difference between a deliberate test and a casual test in the test operation, A function call initiated for a test may lead to multiple sub-function calls at different layers, and the running code outside of the scope of concern will also contribute the coverage rate. Therefore, to ensure the test quality, we have to evaluate the test design level on the basis of evaluating the code coverage rate. The test design level indicator is statistically significant. It analyzes the design scale of the test case and compares it with the total number of branches of the tested code. The more branches of the tested code, or the larger the total amount of code, you should design more use cases to test it.

Secondly, strict white-box testing should begin with coding and continue throughout the entire lifecycle of software products. As long as the code is modified, the test cases need to be supplemented and maintained, white-box testing is not a process, but a long-term and continuous process. It is necessary for us to keep white-box testing in a clear and concise manner to continue smoothly. Otherwise, as an organization, it is difficult to stick to the white-box test. The first test is easier, but after the code is added, it is much more difficult to follow up the white-box test in a timely manner.

On the basis of continuous integration, the 4th-generation white box testing method requires that the testing design be followed up immediately after the source code is modified, and use three signal lights to identify the status of the currently tested system, the first traffic signal indicates whether the current test has passed. If the test passes, the green light is displayed. Otherwise, the test problem must be solved. The second traffic signal indicates whether the current test coverage rate meets the requirements, otherwise, you need to improve the test design. The third signal indicates whether the current test level meets the requirements, and the green light is displayed only when the same conditions are met. Smoothly and continuously promote white box testing, that is, to ensure that the three signal lights are always on. Non-green light is a transient state, which requires that the problem be solved first and then go to the next function development round.

4. select a good tool

Promoting embedded white-box testing is a system project, mainly influenced by three factors: human factors, process factors, and tool factors. These factors interact and depend on each other, it is difficult to implement white box testing.

As mentioned above, excellent employees and common employees in an organization always exist in a certain proportion. The law of 20/80 of the People's Republic of China reflects many natural and social phenomena, as well as product operations, 20% of all employees are excellent employees, 80% are ordinary employees, and excellent employees contribute 80% to R & D, while ordinary employees contribute 20%. Therefore, a white box test is successfully implemented in an organization, there is no need to rely too much on human factors, and more processes should be made to standardize the implementation process, so that ordinary employees can exert value close to excellent employees, and select a good tool, improve overall testing efficiency and quality within the Organization.

We will analyze the relationship between process factors and tool factors. if the process is very standard and the selection of test tools is poor, the white box test can still be implemented successfully. Similarly, if the process is ineffective, but the selected test tool is good, the white box test can also be successful. To evaluate the two, we should identify which factor is more critical to the enterprise, identify the bottleneck, and solve it in a targeted manner. Then, the white box test will naturally be promoted.

Based on experience, most enterprises in China that do embedded products cannot perform white-box testing. The key reason is not to do it or not to do it, but to lack a good tool. The execution of the software process is still the second, because only one group has created a successful case, and it is not difficult to implement it in other groups, and the operation process is easy to build. This point is easy to be verified from the other side. For example, the continuous integration testing and testing practices that have been booming in recent years have seen many successful cases in Java and C # project development, however, there are not many successful projects in C ++, and the success in C is rare. This shows that the lack of white box testing is not a methodology, more tools are used in concert with the methodology.

References:

1. George, B. and Williams, L., "an initial investigation of test-driven development in industry"

2. eztester (China), Wayne Chan, 4th white box testing method overview-Theory

3. eztester (China), Wayne Chan, 4th white box testing method overview-vctester practice

4. eztester (China), Wayne Chan, for example, to do a white box test. Doc

5. Philip M. Johnson, and Joy M. Agustin, "Keeping the coverage Green: investigating the cost and quality of testing in Agile Development"