Auto-generated test and TDD [translated from blog of Uncle Bob]

Auto-generated test and TDD

PostedUncle Bob

Translators: 88250

The test-driven development method (TDD) is now very popular and is widely used by many companies. However, some people are also aware that this will take too much time to write unit tests, so they are always looking for tools to automatically generate tests to reduce the burden of writing unit tests.

This burden is not irrelevant. After using TDD properly, create an application with 45,000 lines of Java code, of which 15,000 lines of code are unit tests. We can quickly calculate that TDD increases the coding burden by 1/3!

Of course, this is a negative analysis. The benefit of using TDD is very important, and its value far exceeds the burden of writing 'extra code. However, 33% of the test code still feels "extra", which also induces people to find a method to circumvent TDD, and this method will not lose any benefits brought by TDD.

Test Generator

Many of my friends want the tool to automatically generate the test code while checking the code. Such tools are rather clever.

They can generate method calls randomly and record the returned results. They can automatically build simulated objects (MOCKS) and stub objects (Stubs) to solve the module dependency problem. They are well-informed algorithms that choose random test data. They even provide plug-ins to help programmers adjust the algorithm for selecting test data to better suit application testing.

The running result of this tool is:Observation Point. When a call with a definite parameter occurs, this tool observes the changes to the class instance variables. It will pay attention to the returned value of the call, the change of the instance variable value, and the call to the stubs and simulation objects. All of these are shown as observations to their users.

The user must view all observation points and determine which are correct, irrelevant, and bugs. After the bugs is repaired, the test will still be run again and again. For testers who use the GUI, this is similar to the playback of the recorder. Once you receive all the correct observations, you can also return to the start of the test and confirm that all observations are still visible.

Such a tool will generate observation points in JUnit testing mode, so you can run these observation points like running the test package. This process is similar to TDD, isn't it? Well, don't worry ......

If you do not make mistakes, the tool is indeed quite useful. If you have a bunch of legacy code that has not been tested, and then generate a JUnit test package to test some of the Code's behavior, how comfortable it is!

Peripheral Problems

On the other hand, no matter how intelligent the Test Generator is, it produces tests that are always more naive than humans write. For example, I have tried to use two good test generators to generate a bowling game program. The interface for this bowling game is shown below:

  public class BowlingGame {     public void roll(int pins) {...}     public int score() {...}   }

You can roll the ball and use the roll method to get the score.

The Test Generator cannot randomly generate available game bureaus. The problem is obvious. A game is a sequence between 12 and 21 throwing balls, with each score being an integer between 0 and 10. The most important thing is that within a frame, the score cannot exceed 10. This restriction is too strict for the random number generator in the current universe.

I wrote a plug-in to instruct the generator to create available games; however, such a guiding algorithm is more logical than the bowling game itself, so it is not smart enough.

The Test Generator encounters a problem that does not belong to any type.Internal AlgorithmProblems, such as call sequence or State semantics. They can generate tests in the perimeter of the class to be tested, but they cannot enter the interior of the class without help.

TDD?

The actual problem is whether the test generated in this way can help you when you use test-driven development (TDD. TDD acts as a tester to drive system development.

First, you write the unit test code, and then you write the tested application code. Apparently, the test generated from existing Code violates this simple rule. Therefore, in some development philosophy, use the Test GeneratorNoTDD practices. But as long as you write a test, who cares about how these tests are written, isn't it? Uh, wait ......

One advantage of TDD is similar to double-entry account registration. The accountant makes one entry twice; one on the lender and the other on the borrower. These two entries follow the mathematical method of independent conditions. Finally, an incredible subtraction produces 0 if the entries on both sides are correctly cleared.

In TDD, programmers state their intentions twice; one is test code, and the other is product code. The two statementsIntentionIs mutually verified. The tester verifies the intent of the product code, while the product code verifies the intent of the test code. This method works only becauseHumanOnly two entries can be created! Although the intent must be stated twice, it is in the form of a supplement. This greatly reduces errors and provides enhanced design-critical insights.

This concept is broken with the test generator because the generator uses the product code as the input to write the test. The generated test is not a repeat of humans, but an automatic translation record. In doing so, humans only state the intent once, so it is difficult to gain insights at record review, and the tests generated do not validate the product code.Intention. People must verify that the observation point is true. Compared with TDD, this is only a negative behavior. Provides insights on few defects, designs, and intentions.

My conclusion is that automatic test generation is not equivalent to TDD, nor is it an effective way to improve TDD efficiency. When you try to generate the 33% testing code, you lose the defect elimination, repeat the intent, and enhance the design insights. You also sacrifice the depth of test coverage because of peripheral problems.

However, this does not mean that the test generator is useless. As I mentioned earlier, I think they can indeed help us see some of the features of the legacy code. However, remember that these tools are not TDD tools. The tests they generate are not equivalent to the tests written in TDD. The benefit of TDD is that testing and production cannot.