Agile question: TDD

Q: Why are few bugs found through unit tests?

A: unit testing is not used to detect bugs, but to prevent bugs. If TDD is used, when the test cases are completed, the product code has not been written, and the bug cannot be discussed.

 

Q: Can I write unit tests to improve the code quality?

A: Some people do not think so. <TDD opinion: quality is a function of thought and reflection, not bug prevention>.
Good, code quality is not necessarily associated with unit tests. methods such as clean room software development can still get high-quality code without unit tests, but this is another problem. perhaps subjectively, the essence of TDD is closer to prompting you to build quality into your thinking,
But objectively, unit tests can still prevent bugs when other conditions are the same.

 

Q: How can unit tests reflect/replace requirements?

A: unit tests may not directly reflect macro requirements,

1. Function Testing and integration testing reflect macro requirements.

2. Unit Testing can reflect the requirements of other parts of the system for the current unit.

From the perspective of text, the name of the test case is the Requirement Description. in other words, if you extract the description from the traditional requirement document and put it in the test code as the name of the test case, you have an executable requirement document.

A function test case written by rspec (do not doubt that it can indeed run ):

It "shoshould show
Welcome Message After login"Do

Login_as_chelsea

Get
: Index

Response. shocould have_text (/welcome
Chelsea /)

End

It "shoshould not show
Welcome Message After logout"Do

Logout

Get
: Index

Response. should_not have_text (// welcome /)

End

Unit test example:

Public


Void

Testshouldbefreefrom2amto5am ()

Throws
Exception {// Direct Business Requirements

...

}

Public


Void

Testshouldthrowexceptionifcannotfindconfigfile ()

Throws
Exception {// requirements from other parts of the system

...

}

Test cases do not exclude requirements documents at the business layer,
The requirement/vision description that highlights the business value is very helpful for quick understanding of the system, but/is just a test case that describes the real system in another way. It has two outstanding advantages:

1. It will not lie, that is, it will always synchronize with the real behavior of the system

2. It is executable, and it can be tirelessly cost-effective at all times,
   Repeatedly asks whether your system meets the requirements

 

Q: What should I do if the demand changes? Isn't there a lot of test cases that need to be modified?

A: shouldn't it be? Does the previous requirement document not need to be modified when the requirement changes? Oh,
Maybe they don't need it, because no one cares about it and it has no impact on the Code. After the first few weeks, the requirement document will be thrown into the configuration library and no one cares about it.

1073x: "This is why many programmers are reluctant to perform unit tests and resist TDD (although they are not the same at all ). It is important to determine what should be described using test cases during development and what can be used. The granularity and quantity of the corresponding test code of a product code are all considerations. "

Yes, this is related to your testing strategy. However, the requirements for unit testing are usually as comprehensive as possible.
Therefore, this is a test design problem. Yes, the test code also needs to be designed, restructured, and
Domain Specific.

 

Q: My unit test compilation links are slow and some conditions are difficult to test, such as insufficient memory or difficult to build environments,
For example, how can we solve the problem of network or database needs?

A: This is an integration test, not a unit test. You must compile and link all the components of the system.
If your test fails, what is the problem?

Usually, unit testing must meet one condition: independent from any other unit, that is, isolation. Implementation means can easily counterfeit dependencies in the testing environment,
And set dependencies to work according to our wishes. An example is that your code depends on malloc to get the memory, and you want to test the memory shortage.
Then we should use a fake malloc in the unit test to replace the real malloc, And we can control the fake malloc to return null.
To simulate insufficient memory. For details about how to do this, refer to some mature "counterfeit" frameworks, such
Mockcpp and so on.

 

Q: My original tests were all run with real code. One test can cover multiple units. Now you have replaced all dependencies,
What if there is a problem with the replaced module? How can we ensure that real code can still work properly after integration?

A: other units have their own unit tests. They focus on themselves. The integration test is the same as before,
If you do not have a unit test, do not perform other tests.

 

Q: Is unit testing a design? How can unit tests reflect/replace design?

A: unit tests reflect local design and are limited to this unit and other units that interact with it.
The unit test mentioned above can reflect the requirements of other parts of the system for the current unit. The so-called design is the division of duties, interaction and dependency between units.

When you try to test a unit, you find that you need to create a large number of other objects and follow the implementation in your mind,
Some objects are created inside the unit and cannot be impersonated in the test environment. At this time, even if you only want to reduce the difficulty of the test, you will force yourself to think:

1. Has this unit done too many things and assumed additional responsibilities, violating the single responsibility principle?

2. Should the dependency be set in the outside world instead of being created internally, so that the dependency can be set as a counterfeit implementation during testing?

Yes, unit tests warn you to think about your design.

 

Q: Is unit testing a design? Is the source code a design? Is it a test design or a source code design?

A: This is actually another angle. The source code is the design argument based on two assumptions.

1. The product of the engineer's work in the design phase, that is, his design,
   It should be able to be implemented strictly and almost identical by different implementers in the implementation phase

2. Software developers are also engineers, that is, software engineers

If we agree with these two assumptions, what products of software engineers can be strictly and repeatedly implemented?
Is it your "design" document in word form? Is it a UML diagram drawn by a CAD tool? None of them, because they are not accurate and there are countless implementation methods, so we cannot strictly discuss them,
Different developers have different implementations. in fact, only the source code can satisfy this constraint. the software design phase is the moment when the software engineer completes the source code, and the software implementation phase,
In fact, only compilation and deployment are left.

 

Q: unit test is a requirement document, while unit test is a design document. How can it be both a requirement and a design?

A: Name and asserted description requirement, Environment Setting Description design...

 

Q: Since unit test describes the requirement, it should be a black box test? Can unit tests always be considered as white-box tests?

A: black and white are relative to the level you observe. Compared with other tests that observe the "System" behavior from the outside and do not involve source code,
The unit test goes deep into the internal observation of the behavior of the box, so it is a white box, while the specific unit test case is still observing the behavior of the "unit" from the external as much as possible, so it is a black box.

 

Q: But the mock technology you commonly use obviously pushes unit testing to a white box.

A: It is a long term, but we can first draw a conclusion: status-based testing over interaction/behavior-based testing,
Although the right side also has great value, we believe that the left side is more stable and has more insight into the system.

Status-based tests describe requirements, while interactive behavior-based tests describe implementation. Compared with requirements, the implementation is easier to change,
Especially under the impact of another practice "refactoring", the test described for implementation will be completely modified, resulting in considerable rework and maintenance costs.

An exception is that interaction is a requirement, and mock is an appropriate choice. For an example, see the final bank API example in <TDD: Tricky Driven Design 3, method>.

In real life, there are some situations, although the use of mock may bring about the maintenance cost in the future,
However, the benefits it brings are irreplaceable. For example, the reduction in the coding capacity of the overall test code in the early stage is particularly evident in the C/C ++ project:

Restricted by the C/C ++ compilation model, you can use the common pre-processing Access Point and compilation Link Access Point technology to access
When implementing stub, be careful to maintain the header file's defense macros, header file names, include path settings for build scripts in different environments, and library path settings.
The way stub is manually written is complicated and error-prone. At this time, an easy-to-use mock framework such
Mockcpp and so on will save a lot of coding and early maintenance work

Almost all mock frameworks support the degradation of mock objects to stub, such
In mockcpp
Defaults () settings, or allowing in jmock 2. In fact, this is my recommended mock usage:
Generally, it degrades to a simple stub and uses its powerful expectation setting and verification capabilities only when necessary.

Usually, unit tests have two accepted constraints to meet:

1. Fast

2. Isolate dependencies.

Again, the conclusion is:
When the fast and isolated dependencies of unit tests are met,

1. State-based black box testing is preferred (handwritten stub or mock degraded stub can be used)

2. Unless interaction and behavior are requirements (all features of the mock object can be used)

 

Q: How to test the private function?

A: change it to public.

I am serious. If you find that private
A function cannot be simply covered by a public function test and requires special tests. This means that your unit may assume too many responsibilities and should be split into a single unit, and open to public
Function.

If C ++ is used, # define private public in the test environment.

If G ++ is used, add-fno-access-control to the test environment.

 

Q: similar to private, some language features with good design intent, such as static, sealed,
Final, non-virtual functions, etc., but always bring trouble to the Code's ease of testing. How should we choose?

A: There is no good way. These language features are the same as the purpose of the test. They are all aimed at improving the code quality and reducing the possibility of errors,
Although things are the same, they are mutually restricted and have different effects.

I think it is time for the industry to seriously consider the test environment. It is best to build the test support in the language,
Some language features designed for the product environment should be closed in the test environment and take effect in the product environment. In fact, many compilers previously supported the release and debug environments,
It is also from the aspect of code quality. It is undoubtedly confirmed that unit testing is more effective than Debug. It is time to keep pace with the times and increase support for test and gradually drop the support for debug.

Before the language itself adds support for testing, we have to find a way to bypass the restrictions on language features in the testing environment, especially for legacy systems,
The Code already exists. For example, for the static function in C ++, the entire unit under test can be # include, or # define static is empty.
Macros represent an indirect layer. In the test environment, this layer of indirect layer is crucial. For other methods, see <working into tively with legacy code>,
<Fake art>.

 

Q: I mentioned how to support "test" instead of "debug". Is there any conflict between test and debug?

A: Yes. If you find that debugging is not required, the testing granularity is too coarse, the steps are too large, and the product code is too long,
You may even be involved in too many units to undermine the isolation of the test. debug is the result of unclear code logic and hard to assert behavior. You can use the test to help you locate the error.

 

Q: I know that to add new features to legacy systems, we need to first write tests to ensure the original behavior of the system, but the legacy code is huge,
I don't even know the current behavior of the system. What should I do?

A: feature test: Maintain the code behavior test, obtain the current running result, and fill in the test to obtain the current system behavior.
In fact, the test can be divided into two types: Try to explain the target to be achieved, or try to maintain the existing behavior in the Code; After feature implementation, the former will be converted to the latter. for more information, see <working
Extends tively with legacy code>

 

Q: Are there mature practices for using TDD or unit testing on legacy systems?

A: or <working
Extends tively with legacy code>, or <restructured a project based on a large legacy system>

 

Q: I often talked about C ++ or other object-oriented languages, but I didn't mention C. How should I use TDD in procedural languages? What's different?

A: basically the same, and TDD is used in procedural languages, the object-oriented design may be exported.
For example, if you call a function directly, you have to use the replacement during compilation or link replacement to access the false implementation. This is actually troublesome, so it may prompt you to use the function pointer.
To facilitate replacement in the test environment. over time, you will find that a group of function pointers related to concepts appear, so bind them with the data they operate on and define a struct,
This forms an object style. Of course, this may make your code more complex, which needs to be selected in practice.

It is also possible that in a procedural language, you think TDD is not very helpful to the design, and the test cases are also boring, that is, testing a branch,
Return value or something. Yes, the logic is hidden in the Branch and return value. If you are used to procedural thinking and do not intend to change, the impact of TDD on design is more reflected in dependency management,
For example, the division of duties of header files and compilation units. If you mix functions with different responsibilities in a compilation unit, it is difficult to implement methods such as link replacement unless you choose a similar
The mockcpp framework does not need to be replaced by links.

 

Q: How do testers arrange TDD? Is it necessary to enter the project team at the beginning?
But there was no product code at that time. What should I test?

A: Yes. It is necessary to enter the project team at the beginning, not because of TDD. Yes,
The tester was not measurable at the beginning. It does not mean that they did not work.

TDD is a development method and an activity attended by developers. The effect is to document your needs in an executable form,
Forces you to differentiate duties and isolate dependencies to drive your design, and weave security nets to eliminate bugs in the cradle state to prevent escape. however, traditional testers try to find bugs that have escaped. both activities are necessary,
There is no conflict between them and they complement each other.

So what do testers do before new features are developed? In addition to writing test cases in advance, both automated and non-automated,
An important activity that requires testers to participate is to participate in the formulation of feature acceptance conditions. previously, developers often coded according to their own understanding, and testers tested according to their own understanding until the development was completed,
During the test, inconsistency is discovered, disputes are generated, and arbitration waiting for business analysts (if Lucky) or administrative directors (if the development process is chaotic) is blocked.
The solution is to discuss with the business analysts, testers, and developers before developing new features to reach an agreement on the acceptance conditions and form a record,
Then the tester and the developer write the test and implementation separately.

 

Q: There was a phase before, that is, after the development of a group of related features is completed, the tester took over the test. After several rounds of bug fixes,
The product can be released in a stable manner. Now, testers are involved in each iteration in advance to test a single feature. How can we ensure the quality of product integration?

A: as before, there must be an integration test phase, depending on the current quality of the product.
It does not mean that with the iteration level, a single feature-level test does not need to be released. There is no conflict between the two.

 

Q: What are the advantages of testing staff entering iteration ahead of schedule?

A: identify the problem as soon as possible and reduce the cost of fixing the error. There are several methods,
First, we discussed the acceptance conditions with the business personnel and developers, so as to prevent rework caused by comprehension deviation. Second, the Development completes immediate testing and immediate feedback upon problems found,
In this way, developers are still impressed with the code and can quickly locate and fix errors. in this way, there will be fewer bugs in the final integration test phase, which will shorten the final integration test time and ensure more stable product release.

 

Q: sometimes subsequent features will affect the previous features. In the iteration process, the tester only tests a single feature,
How can we ensure that previous features still work?

A: several means. test should be automated as much as possible so that continuous integration can be achieved. Then, dependency management should be done well. Every time a new feature is completed,
We should be able to find other features affected by it and see if some integration tests should be added.

 

Q: Sometimes, when a developer completes a feature, it is close to the end of iteration. The tester does not have time to perform a full test. What should I do?

A: In the next iteration test, when calculating the development speed, you should only calculate the features that pass the acceptance of testers in this iteration,
Features that are only completed by developers and not fully tested by testers are excluded. this situation is inevitable. however, we can use some means to make the test and development more synchronous and minimize the lag,
This includes making testers work more closely with developers and automating test cases as much as possible.

 

Q: I still think the workload of testers is not full during the Development iteration process.

A: If this is not what you feel, but the facts, and all the work that the previous Tester must do is still not full,
Congratulations! You can save some testers and do other things. however, it is not recommended that testers should not work for both teams at the same time. this greatly increases the communication cost. you will often find that,
When your developer is looking for help from a tester, but cannot find anyone, your team is blocked, and the context switching of the tester is also painful.

 

Q: You said that the acceptance test should be written by the customer, but it is impossible for us.

A: The acceptance is of course the acceptance by the customer. Theoretically, this is beyond doubt and must occur in all walks of life.
It is only specific to the writing and execution of test cases. Whether it is automated or not, you must master certain technologies and think carefully. It takes special time, the customer may not be able to meet these conditions at the same time,
If we try our best to overcome this problem, we have to make up for the distortion of the Greater generation of workers through more adequate communication. At this time, business analysts and testers should work together to complete the preparation of the acceptance test.

 

Q: You said that the ratio of your previous project product code to the test code is about. This is not a case of adding 3 to the White List.
Times the workload?

A: It is a three-fold increase in the amount of code rather than the workload. It saves you dozens of people a few months of pre-designed work,
It saves you the time to design each module in detail and write hundreds of pages of detailed documents for it, saving countless sleepless nights of debugging time, it saves the workload of fixing hard-to-count bugs in the integration phase,
It even reduces the number of your product code, removes a large number of duplicate code, and removes a large number of overly-designed complex code, making your code easier to understand, it is easier to add new features and locate bugs,
That greatly reduces the workload of maintenance over several years? But this is TDD.
And the benefits of agile development (if you already have practice) and Vision (if you are still waiting to watch)

 

Q: We also perform unit tests, but first write the product code and then write the test. Do we have to test TDD first?

A: There is nothing to do. It depends on what you want. TDD only forces you to build quality in your mind in a verifiable way,
A long-term test will first repeat the quality of your thinking, and subsequent unit tests will only be a treasure follower.