Vectorcast-Control the quality of the product by ensuring the integrity of the test

Problems with software Testing
There is a maxim that says, "If you are not prepared beforehand, it means that you are prepared for failure." "If you apply this metaphor to software testing, you can say," without testing, it means that the test failed. " The cost of software failure is enormous: for example, the oven or dishwasher does not work properly, the product may lose its customers, and the medical device fails, and the patient may lose his life. Many organizations want to improve software quality through rigorous development processes and static analysis, but there is no guarantee that software applications will not have problems.
This article is not meant to provide a recipe for developing software that is completely free of problems. Instead, explore what development strategies can help improve software quality.

Figure1: The cost of fixing problems in a product is much higher than finding problems early in software development(Capers Jones,software Assessment, benchmarking and best practices, Edisenwies Publishing House, the) "Hockey Stickthe chart (Figure1) is the most commonly used chart in software development history. This diagram illustrates the cost of repairing problems that exist in deployed applications, much more expensive than fixing them during the code development phase. It is well known that rigorous testing processes are beneficial to the entire software development process, so many companies try to measure the integrity of their tests. But most of the time, when organizations try to improve the quality of their software, the first question is "what will this do to the timing and the workload of the developer", rather than "how we should measure the integrity of the test." "

The first step that all software development organizations need to do to improve the quality of their software is to understand what is being tested today. One of the simplest ways to measure the integrity of existing tests is to increase code coverage analysis during testing. Code coverage metrics can be applied throughout the development lifecycle and do not require significant time and money, and do not adversely affect the scheduling of projects. By organizing existing test processes, code coverage analysis is able to obtain available test validation data.

What is code coverage analysis and why is it important?
Code Coverage analysis refers to the interweaving of the program's source code with the "bread crumbs" that generate the tracking data during the test of the application. This data is post-processed and shows which source code was executed during a certain or a set of tests. Code Coverage Analysis helps developers to answer key questions such as:

• Whether all parts of the application have been tested
• Whether there is a dead code that should not exist in the program

Automated Code Coverage analysis is important because it has the following advantages:

• Easy to integrate into normal software development processes
• Does not affect the test programs and processes in progress
• Options can be flexibly deployed to cover all stages of testing

measuring the importance of test integrity
One thing that seems to be self-evident is that the parts of the program that are not rigorously tested are the most susceptible to problems. There is no way to really improve software quality unless you can find places that are not fully tested. A more typical example is that some organizations, when using code coverage analysis for the first time, are surprised to find that only 70% of the program code is executed under the current test program. It is possible that the remaining 30% of the code is non-critical, error-handling, or edge-case, but it is also possible that these untested code is a critical part of the program, and only the team that developed the test program ignored the code.
If there is no measure, the validity of the test cannot be estimated. Coverage analysis is the best way to measure the integrity of your tests.

testing the original codebase (Legacy code base)
Almost all applications use the old code, but it is difficult to understand the functionality, expected effect, and quality of the legacy code. If you modify or refactor old code that does not have a relevant test architecture, you will certainly break some of the features you are using. So how to improve the quality of the original code base? The first step should be to include code coverage analysis during system testing or functional testing. If you perform coverage analysis at the same time as testing, you can separate the well-tested parts of your program from the ones that are under-tested. Developers can confidently modify those parts that have been fully tested, but they need to be very cautious if they modify the parts that are under-tested.

Once the test results have been obtained, the new functionality can be tested based on coverage information after the new feature development is completed. This way, after a period of time, the code base will be fully tested, the quality will be improved, and easy to maintain.

If you do not refactor the code base, the codebase becomes unmanageable, but refactoring work cannot be completed without a full test.

focus on test resources
Some software applications may not need to reach 100% coverage. Because these programs may be less critical, or only a fraction of the functionality is often used. Even so, you need to know which parts of the program have been tested and which ones have not yet been tested. This is because it facilitates the focus of testing resources on where they can produce the most benefit, and provides important test information, such as test effectiveness metrics. After all, if the user cannot enter the URL in the address bar, then even if the advanced options in IE browser is powerful, it is meaningless to the user.

What should be tested is as important as testing itself.

Apply coverage analysis to the entire life cycle
One of the advantages of coverage analysis is that it can be applied throughout the development life cycle (from unit tests conducted by the initial developer to the final Test by the tester). In all cases, the goal of coverage analysis is to assess the effectiveness of test activities. If coverage analysis is performed at each test phase and all analysis results are merged into a single report before each release, the problems that are encountered when the integration starts are reduced and the release date is not affected, and the problems that the customer may encounter will be greatly reduced.

business case with coverage analysis
While many people agree that producing high-quality products is a good goal, high-quality delivery costs will be correspondingly higher. Moreover, high-quality products need to spend more time and money. Every businessman evaluates these expenses when it comes to business goals.

We believe that the use of coverage analysis can bring tangible and significant business benefits, including:

• Improve the user experience
• Make code libraries easy to maintain
• Enhance competitive advantage

Improve the user experience
Many records record that in the past 30 years, the basic industry has undergone a quality revolution. Oddly, unlike other products in crowded markets, software does not need to comply with the same stringent quality standards (except for high-security software) with similar products. In fact, when the product is delivered, its software flaw is a common problem, and the customer already knows to restart when the product stops working. But if there is a problem with the software, it often fails, reducing the brand value. The company's reputation, carefully crafted, may be disrupted by the release of an unqualified product. Today's customers are more powerful than ever, and they can now use social media tools to instantly comment on the product, provide feedback and quickly reflect the quality of the product.
By using coverage analysis to assess the integrity of your tests, you can ensure that all of the functionality in the program has passed the test before the customer uses the product, and that the customer is not the first to perform a function in the program.

Make code libraries easy to maintain
Now that the software application is getting bigger and longer, it means that the code that is being built can take a long time and will be used in many environments as the program expands. Development organizations must be confident that their code base is fully tested. Because only then, when the team develops the new application, it can guarantee that the new function will not break the existing function, the whole system can achieve the expected effect.
Code Coverage analysis exposes vulnerabilities in the testing process. After a period of time, the integrity of the test and the quality of the software will be improved by constantly discovering vulnerabilities and resolving problems.

Enhance competitive advantage
In 2012, IBM launched the Global CEO survey, involving more than 400 business executives and it executives, to explore factors that affect the competitiveness of organizations in the current market. Executives attribute technology to the most important external factor affecting organizational competitiveness, and more than 50% of CEOs believe that software development is a key driver for competitive advantage. (2)

Figure2: High-quality software development is a key driver for gaining market competitive advantage (sourceIBM)

In the software development Lifecycle, when new features are added or fixed, an automated, complete, repeatable test process can help reduce the number of regressions. This will shorten the release cycle and speed up the market. In today's highly competitive environment, the time of the market is directly related to the product will be in the industry leading position or failure status. in this increasingly popular market, in the development process of software quality in the first place, is a very sensible business strategy.

The Ideal Coverage analysis tool
What if the developer is going to provide information for this test tool? They will undoubtedly list many features, but if condensed, the following key features can be summed up:

• Easy integration with existing architecture and test steps

        -  easy to build a full version of the application system

• Virtually no impact on existing test activities

       - Testers will not find that coverage metrics are in progress

• Supports testing at every stage

-A unified coverage view of all types of tests is extremely important

• Trend Report

-Report How the code base is extended, and whether code coverage is synchronized

• Combined with a variety of compilers

- All compilers have a specific extension to support the appropriate language syntax, which must be supported.

• Integration with requirements management tools

-If there is a formal need, then the test and coverage needs to be mapped with the requirements

• Full command-line interface

- scripting to get data and reports is the key to automation

• Scalable to support large code

    - storing the metrics of code coverage in a relational database is the key to achieving scalability

Choosing the right test tool is extremely important for developing high-quality applications.

Conclusion
Developing high-quality software applications is a gradual process, and there is no such thing as rapid quality improvement, no automated solutions, and no other shortcut to achieve this goal. As engineers and management teams strive to develop high-quality applications, there are several basic strategies to keep in mind:

• Without metrics, you can't measure the effectiveness of your tests, and coverage analysis is the best way to measure the integrity of your tests.
• If you do not refactor the code base, the codebase becomes unmanageable, but refactoring work cannot be completed without a full test.
• What should be tested is as important as testing itself.
• By using coverage analysis to assess the integrity of your tests, you can ensure that all of the functionality in the program has passed the test before the customer uses the product, and that the customer is not the first to perform a function in the program.
• Code Coverage analysis exposes vulnerabilities in the testing process. After a period of time, the integrity of the test and the quality of the software will be improved by constantly discovering vulnerabilities and resolving problems.
• In this increasingly popular market, in the development process of software quality in the first place, is a very sensible business strategy.
• Choosing the right test tool is extremely important for developing high-quality applications.

Conclusion: In order to ensure the quality of software, it is necessary to carry out a complete and effective test, and the best way to measure the integrity of testing is to carry out coverage analysis.

Vectorcast-Control the quality of the product by ensuring the integrity of the test