Misunderstanding of software testing

Testing has always attracted much attention in the software development process. Even in traditional software engineering, there is a clear and independent testing stage. With the frequent emergence of software crises and people's further understanding of the nature of software, the status of testing has been improved as never before. Testing is not only limited to one stage in software development. It has started throughout the software development process. people have begun to realize that the earlier the test starts, the more frequent the test execution, the lower the overall software development cost will be. Extreme Programming pushes the test to the limit. All software development activities should begin by writing the test code first.

However, compared with the popularity of the word "test", the work on testing education is far from enough. Many articles on testing are aimed at the use of a certain testing tool, testing tool manufacturers often boast of their commercial purposes. In this way, many software practitioners may easily fall into some misunderstandings, leading to the failure of testing to play an effective role in their software development projects. The following sections describe some typical mistakes, and describe some design ideas behind the test, hoping to serve as a reference.

One of the misunderstandings is that the test tool is used for effective testing.

This misunderstanding can be said to be a common problem. The CASE tool in almost every field will bring about this problem when it first appears, for example: if a software development team uses the Rational Rose tool to draw UML diagrams in software development, they may claim that they have adopted the object-oriented method, no matter how procedural their design and code are.

The same is true in the field of testing. A software development team often thinks that as long as they use a software testing tool, they should be able to obtain the benefits of testing. This idea is of course wrong. This is because the software or module should be testable before it can be effectively tested. Testability is an internal attribute that reflects the quality of software. It does not make the tested software testable because you have used a testing tool for testing. If the software to be tested is not testable, the benefits of testing with expensive testing tools are minimal.

Coincidentally, there is a natural correlation between testability and other aspects of good software quality, A software with testability must be a software with strong cohesion, weak coupling, clear interfaces, and clear intentions. untested software often has strong coupling and chaotic logic. More information about testability is beyond the scope of this article. For more information about testability, see relevant literature. (More in-depth information about testability is provided in the references attached to this document ).

To truly obtain the huge benefits brought by testing and maximize the efficiency of testing tools, the key is to make the software itself highly testable. The acquisition of such capabilities is a gradual process and cannot be achieved overnight. The most important thing is to continue to practice, learn some excellent experience, and constantly reflect on it. Efforts must be made to obtain good results. This is an everlasting truth. The test-first practice in extreme programming is a good starting point. For more information, see [3].

You only need to choose a test tool to meet your needs and automatically run the test case. Do not blindly pursue complex functions and unnecessary flexibility. For most projects, some well-known open source code testing tools are enough, such as Java unit testing tools JUnit and C ++ unit testing tools cppunit. In terms of acceptance testing, there are currently no general testing tools to meet the needs of various aspects, but the script language is used, it is not difficult to develop an acceptance test tool that suits you step by step. one sentence: only by improving the internal quality of your team can external tools play a role.

Misunderstanding 2: there are too many things that cannot be tested

In the field of software development, some things seem to be more difficult to test than others, but far from being unable to test. In most cases, it is because the tested software itself did not consider testability issues during design. However, this non-testability is not caused by the internal tight coupling of the tested software, but is too tightly coupled with some difficult parts of the outside for testing, as a result, it is difficult to test the tested software. These hard-to-test components are common: graphical interfaces, hardware, databases, and so on. The following uses a graphical interface as an example to describe it.

If a software module can trigger its application logic only through a graphical interface, a graphical interface must be used for testing the software module. However, it is difficult to test the GUI. It seems difficult. Let's consider from another perspective. In fact, what we really want to test is the application logic of the software module, rather than the trigger logic of the graphic interface. If we can well encapsulate the tested software modules and define well-defined services to provide interfaces, we can test them through the interfaces provided by the software modules. This allows you to bypass the graphic interface that is difficult to test. The reason why the above software module is difficult to test is that the application logic of the software module and the graphic interface are coupled in the design. Separating these two logics not only makes the software module easier and more effective, but also makes the software module highly reusable and portable.

What should we do for a graphic interface that is not easy to test? The principle is very simple. If something is hard to be tested, let it do something that will never go wrong, and strip out the logic that may go wrong and put it in a testable module. For the graphic interface, it just maintains a very thin graphical interface logic. Its job is to simply forward user requests to the software module that actually processes the request (generally called Application facade ). The forwarding logic is simple enough that it will certainly not go wrong, so we do not need to test it. For more information, see [5].

If you can write the test code first during software development, it will force you to consider the problem from the perspective of usability and testability, in this way, you will focus on the high-level abstraction and responsibilities of the software module. This will define a clear module interface that clearly reflects the intent. In addition, in order to make the test possible, you will take the initiative to remove the coupling that leads to poor testing. This result not only achieves testability, but also produces a better design and system architecture.

?
However, there are some difficult tests and it is difficult to execute some very simple logic, such as BSP in an embedded system (board-level support package ). The language, environment, and features used to develop them make it difficult to test. The hard test mentioned here is actually a test cost problem. Specifically, it takes more time and effort. This requires you to balance the cost and the benefits of the test. If the benefits brought by the cost (less debugging, maintenance, and release costs) are huge, the cost is very worthwhile.

Misunderstanding 3: Test code can be written at will

Everyone must know that the test code cannot be written at will, and it is not a casual attitude when writing the test code, however, the test code you have compiled and the code running shows randomness and disorder. Because you may not have figured out the true intent of the test.

I have seen such a case about acceptance testing. Acceptance testers use expensive commercial testing tools to test a software with a graphical interface. The test method is to write a test script to drive random mouse clicks on the graphic interface (of course, each click is directed to the area where events can be received on the graphic interface ), then wait for the software to be tested to crash. Of course, this test method can be used as an acceptance test, but it is by no means the only one. More important content is ignored.

The purpose of the test is to check whether the software system meets the requirements. Therefore, your test code must clearly express this point. In the above case, if the tester really starts from the user's needs, then the test script he wrote will certainly not be like that, however, each test case clearly depicts a user's needs and then checks whether the system has implemented the user's desired functions. Such a test has a clear purpose and is the most effective test. Compared with isolating the interface logic from the application logic and using test cases that clearly express user requirements for testing, the above test method cannot be left blank.

Some Incorrect testing methods are often seen in unit tests for classes. Many testers often perform tests for each specific implementation detail in the class. The specific implementation details in the class are easily changed, especially in rapid iterative development. Once the implementation details of a class you test have changed, the original test code must be changed accordingly. Otherwise, the meaning will be lost. The cost of such frequent changes is huge. Class is an abstraction that reflects the cohesion of a higher level. It should have clear responsibilities and well-defined service interfaces, its responsibilities and external interfaces are much more stable than the internal implementation details, and we need to verify whether this class has its responsibilities. Therefore, when testing a class, we should verify whether the class has implemented its responsibilities rather than other interfaces. In this way, the written test code is much more stable and effective.

For more information, the main reason for Testing implementation details is that classes are implemented before testing. If you first implement the class function and then test the class, the subconscious will involuntarily want to verify some implementation details of the completed class. If you can write the test code for this class before writing the actual class, the situation will be very different, because this will force you to consider the problem from the perspective of the class users. The result is that the focus is on the ease of use of the class, the responsibility of the class, and the interface of the class to provide services, rather than some implementation details.

In short, the writing of the test code should be done at the level of whether the tested object meets the needs, rather than other aspects.

Misunderstanding 4: unit test and acceptance test are no different

Like misunderstanding 3, many may not admit this. However, they cannot clearly tell the difference between the two. In this way, they will be confused when writing the test code. This section attempts to provide some principles to distinguish between unit test and acceptance test.

We also take buildings that are often used in analogy with software as an example. First, we will give you a perceptual knowledge. The unit test can be analogous to the inspection of a building by a quality control personnel. His focus is on the internal structure, foundation, frame, and vertical walls of the building. His detection is to ensure that all parts of the building are normal and safe. In other words, it is to ensure that the construction meets the quality standards on the building. The acceptance test can be compared to the user of the building to inspect the building. First, he believes that the building meets the prescribed engineering quality, which is guaranteed by quality control personnel of the building. The focus of user attention is the feeling of living in this building. He cares about the appearance of the building, the size of each room, the location of the window, and whether it can meet the needs of the family. Here, the user of the building performs the acceptance test from the user's point of view. The quality control personnel of the building Perform unit tests from the perspective of the builders.

The difference between unit test and acceptance test depends on this angle. They are tests on different aspects of the system, and they complement each other. No matter how smart we use in system building, no matter how flexible our system is, our products must first be available, otherwise, what we do is a waste of time. In this regard, acceptance testing is more important than unit testing.

Taking the case given in the above section as an example, the test method used in the case is only from the perspective of system robustness. Even if the system never crashes, it cannot prove that it is an available system. Because the test is not from the user's perspective, the tester should work with the user to compile the acceptance test. Unit testing ensures that we do the right thing, while acceptance testing ensures that we do the right thing.

There is no general standard for the clear division between unit test and acceptance test. You can increase your experience in this area only through continuous practice. The more tests you perform, the clearer you will feel the boundaries between unit test and acceptance test. The following provides some guiding principles that may be helpful when writing test code.

If a unit test is to span the boundaries of a class, it may be an acceptance test.
If a unit test becomes complex, it may be an acceptance test.
If a unit test often changes as user needs change, it may be an acceptance test.
If a unit test is harder to compile than the code to be tested, it may be an acceptance test.

Conclusion

Testing is used to ensure the efficiency of the software development process and the high quality and availability of the developed software products. Software development itself is a very difficult task, which also determines that effective testing is not simply done by relying on some testing tools. While using tools, we should also strengthen training and education on testing so that everyone can have a correct understanding of testing. Only in this way can we use tools more effectively and efficiently so that testing can truly play its due role. I hope this article will help you in your testing.

References

Extreme Programming explained: embrace change, Kent Beck, 1999
Agile Software Development, principles, patterns, and practices, Robert C. Martin, 2002
Test driven development: by example, Kent Beck, 2002
Refactoring: improving the design of existing code, Martin Fowler, Kent Beck, 1999
Testing things that seem hard to test, Robert S. Koss, 2001