Use the IBM Rational test philosophy to successfully build a test team

Level: elementary

IBM,

June 01, 2004

This document describes the concepts of IBM Rational in software testing and the ratioanl testing product solutions. You can learn how to use the IBM Rational testing concepts to successfully build a testing team.

1. Problems in traditional software testing

Testing is the most important part in all software development processes. In the process of software development, on the one hand, we need to test the activities to verify that the software developed meets each of the features described in the software requirements in terms of functionality, the performance meets the load pressure and response time and throughput requirements of the customer. On the other hand, for the market and customers, the development team must also meet the requirements for release of software as soon as possible within the budget.

The traditional software testing process generally involves a small number of unit tests during the software development process, and then a large number of tests are carried out at the end of the entire software development stage, includes integration testing of functions and performance and system testing. As software projects become more and more complex, the traditional software testing process inevitably brings the following problems to our work:

• Problem 1: It is difficult to control the project progress and increase the difficulty of project management.
  1. As shown in, a large number of software errors are often discovered only during system tests at the end of the project. The time it takes to solve the problem is hard to predict and the project progress is often uncontrollable, at the same time, throughout the software development process, the project management personnel lack the understanding and control of the software quality, which makes the project management more difficult.
  Figure 1. Problems in the traditional test process
  
• Question 2: weak Project Risk Control Capabilities
  Project risks can be reduced when the project development is late. It is often after a system test that the design can meet the functional, performance, and reliability requirements of the system.
• Question 3: the software project development fee exceeds the budget
  During the entire software development cycle, the later the error is detected, the higher the cost of fixing the unit error. Second, the solution to the error delay will inevitably lead to a sharp increase in the cost of the entire project.

  Figure 2. Problems in the traditional test process
  

Back to Top

2. Solve Traditional testing problems with the best successful IBM Rational Software automated testing experience

Three best success experiences of IBM Rational Software automation testing technology core: early testing, continuous testing, automated testing, on this basis, we provide a complete software testing process and a complete set of software automated testing tools, so that we can ultimately achieve: A test team, based on a complete software testing process, complete software quality verification with a complete set of automated software testing tools.

2.1 Successful Experience 1: early testing

Early testing refers to the idea of completing various software testing tasks as early as possible through various software engineering technologies throughout the software development lifecycle. IBM Rational provides early testing software engineering technologies in the following three areas:

First, the entire test life cycle of the software is basically the same as the development life cycle of the software, as shown in figure 3, that is to say, when the demand analysis is basically clear, we should carry out the software test plan based on the results of the requirement analysis and the entire project plan. The design of the test case should be completed along with the analysis and design process; after the first release of the software, the tester should immediately implement the test script based on it, and execute the test cases based on the test purpose in the test plan to evaluate the test results. In this way, we can monitor the quality of the project in real time through various test indicators to improve the control and management capabilities of the entire project.

Figure 3. Software Test Lifecycle

Secondly, through iteration, software development divides the entire software development lifecycle into multiple iteration cycles, and tests are carried out in each iteration cycle, this greatly advances the software system testing time, which can greatly reduce project risks and project development costs.

Finally, IBM Rational's early success experience is also reflected in its expanded Division of the traditional software testing phase from unit testing, integration testing to system testing and acceptance testing, the entire software testing is divided into two stages: developer testing and system testing. As shown in figure 4, it is duty-bound to extend the software testing to the entire development process. Improve software quality as early as possible and reduce software testing costs by testing the time in advance.

Figure 4 division of the test phase by IBM Rational Testing Method

2.2 successful experience 2: Continuous testing

Successful testing experience continuous testing is derived from the iterative software development model. In the iterative method, we divide the development goals of the entire project into some small goals that are easier to accomplish and achieve. These small targets all have a clearly defined stage evaluation standard. An iteration is a series of development activities to accomplish certain stage goals. Before each iteration starts, an iteration plan should be formulated based on the current state of the project and the stage objectives to be achieved, in addition, each iteration includes a series of development activities such as requirements, design, coding, integration, and testing, and some new system functions will be incrementally integrated. Through each iteration, we generate a runable system. Through testing the runable system, we can assess whether the iteration has reached the expected iteration goal, then, the goal of the next iteration is determined based on this. It can be seen that we conduct software testing activities in each iteration cycle of iterative software development. The completion of the entire software testing is achieved through continuous incremental testing and regression testing in each iteration cycle.

As shown in figure 5, the successful software testing experience of Continuous testing can not only continuously improve the software quality and monitor the quality status, but also make the early implementation of system testing possible. This effectively controls development risks, reduces testing costs, and ensures project progress.

Figure 5 successful IBM Rational testing experience: Continuous testing

2.3 successful experience 3: Automated Testing

To achieve early and continuous testing during the entire software testing process, we can say that a complete testing process is a prerequisite and an automated testing tool is a guarantee. IBM Rational's successful automated testing experience mainly refers to the use of software testing tools to provide complete software testing process support and automated implementation of various tests.

In order to allow various software testing teams to perform better tests, IBM Rational also provides a complete set of software testing processes and automated testing tools in addition to the successful testing experience, enable the software testing team to complete the entire test task without being forced.

Back to Top

3. IBM Rational Software automated testing tools

In IBM Rational's software automated testing solution, what we have been striving for is the unification of testing tools, testing success experiences, and testing processes, each successful test experience and test process described above can be completed through rational testing tools and perfect integration between tools.

IBM Rational's automated software testing tool, 7, has the biggest feature of implementing the test management process through a complete set of software testing tools, it also covers automated testing requirements for functional testing, performance testing, and reliability testing. It integrates testing resources through integration between tools to help the testing team achieve successful testing experience of IBM rational.

Figure 7. IBM Rational Automated Testing Tool

Back to Top

4. IBM Rational Software Testing Process

The IBM Rational software testing process not only includes the complete software testing process framework, but also supports test management tools embedded with software testing processes.

4.1 IBM Rational software testing process framework

IBM Rational Unified Process (hereinafter referred to as RUP) provides a complete testing process framework, which can be used by the software testing team based on the actual requirements of business development, customizes the software testing process used by the team. The Software Test Procedure 6 in RUP is as follows:

Figure 6. IBM Rational Software Testing Process

Each test stage is described as follows:

• The purpose of preparing a test plan is to determine and describe the tests to be implemented and executed. This is accomplished by generating a test plan containing test requirements and test policies. A separate test plan can be developed to describe all different test types to be implemented and executed, or a test plan can be created for each test type.
• The purpose of the design test is to determine, describe, and generate the test process and test cases.
• The purpose of the Implementation test is to implement (record, generate, or compile) the test process defined in the design test. The output workpiece is a computer-readable version of the test process. It is called a test script.
• The purpose of the test is to ensure that the entire system runs as intended. The system integrator compiles and links the system in each iteration. Each iteration needs to test the added features and repeat all test cases tested in previous versions (regression testing ).

The purpose of the evaluation test is to generate and deliver the Test Evaluation summary. This is accomplished through review and evaluation of test results, determination and recording of change requests, and calculation of major test evaluation methods. The Test Evaluation summary provides test results and main test evaluation methods in an organized format to evaluate the quality of test objects and test processes.

4.2 automated software testing process using the IBM Rational Software Testing Management Platform

IBM Rational builds the software automated test management platform tool testmanager on the basis of the test methodology of the RUP. It is perfectly integrated with the test requirement management tool requisitepro and the defect tracking tool ClearQuest, it manages the entire software test lifecycle and helps the software test team quickly establish a Software Test Platform and test management process, so that the software test team can quickly possess the following capabilities:

• Testmanager provides a core platform for test management, it integrates management activities throughout the test life cycle, including test requirements, test plans, test design, test implementation, test execution, test result analysis, and automatic generation of test reports. At the same time, a variety of test suite, test case, and test script are organized to facilitate regression testing.
• Testmanager complies with the standard test process of RUP, enabling testers to follow the unified test management process on a unified test management platform, complete all-round quality tests including product functionality, reliability, and performance.
• As an integrated solution, rational testmanager works with other rational tools to provide traceability from test requirements to software test process management, defect tracking, and test result evaluation, it facilitates testing management personnel to monitor software testing processes and collect and analyze various quantitative indicators related to software quality.

4.3 automated functional and performance testing using IBM Rational Software testing tools

Another feature of IBM Rational's automated software testing tool is that through testmanager + robot, you can complete both functional and performance tests while implementing the test management process, this greatly shortens the learning process of the test team on the tool and improves the ease of use of the tool.

4.3.1 automated functional testing of software

The function test is mainly centered on the Windows graphic interface, character terminal, and browser interface. The client can be software compiled by VC, VB, Pb, Delphi, various character terminal software, or run the browser Microsoft Explorer and Netscape, automated function/regression testing is achieved through automatic recording to form a test script.

The goal of IBM Rational's functional testing solution is to make functional testing simpler, effective, and repeatable, so as to quickly improve the functional testing capabilities of the software testing team. It has the following features:

• It can easily develop applications and character terminal software in various environments (IDES, complete all test processes including test plan, test design, test implementation, test execution and test result analysis.
• Allows you to easily record or compile various functional test scripts for automated function/regression testing.
• Use the data pool to conveniently test the function of driving large volumes of data;
• It can easily complete the distributed function test and test multiple test platforms at a time;
• It can automatically cover functional test requirements and ensure that the application meets each business requirement of the Product Specification and test plan;

To improve support for functional testing of Java and Web applications, the IBM Rational functional testing solution also provides IBM Rational xde tester, it is mainly used for functional testing of application software developed based on Java and Web on Windows and Linux platforms, it is especially suitable for teams that use development platforms such as IBM WebSphere Studio, eclipse, and rational xde developer for software development. Its three most important automated testing features are:

• Professional automated test Script Creation environment: the test platform is extended and embedded into the IBM WebSphere Studio, eclipse, and rational xde developer development platform, which unifies the test and development environment;
• Test scripts have strong flexibility and maintainability in regression testing: scriptassure is a set of advanced capabilities provided by IBM for testing Java and Web applications, it can help create flexible and reusable test scripts, greatly improving the maintainability of the scripts. Object mapping provides a core object library for testers to modify and verify the objects under test in the program, all related test scripts are automatically updated based on modifications. You can set the property set of the objects to be tested in the tested program, so that changes in a small number of object properties will not affect the normal playback of the test script. In addition, you can create verification points for dynamic data and use regular expressions to verify dynamic data;
• Powerful testing scripting language: using standard testing scripting language Java, You can take full advantage of the advantages of industrial standard languages for testing.

4.3.2 automated stress testing of software

The IBM Rational stress testing tool aims to quickly improve the performance testing capabilities of the software testing team, including load testing and stress testing. The rational performance testing solution allows you to easily and flexibly simulate various load models and perform performance tests with the goal of finding response time bottlenecks, system throughput, and maximum concurrency virtual users. Including:

• Using teststudio, you can manage the test requirements, test plans, test design, test implementation, test execution, and test result analysis throughout the test lifecycle;
• Using test suite in teststudio, you can easily complete various requirements of the load model for stress testing, including:
  • Establish a complex scenario model;
  • Accurately simulate Timing Control of complex loads;
  • Load Analysis Based on transaction;
  • Create a target-oriented transaction load model, for example, 100 transactions/second.
  • Response time accurate to 1/100 seconds;
  • Supports simulation of different IP addresses of different virtual users;
  • Accurate baud rate Simulation;
  • Using teststudio, You can conveniently observe various metrics during stress testing;
  • Using teststudio, you can easily analyze the stress test results and generate various results reports;

4.4 automated software reliability testing and unit testing using IBM Rational Software testing tools

IBM Rational Software Testing Tool purifyplus is mainly used to help the software testing team quickly improve the unit testing and reliability testing capabilities in the short term. Its main features are: quick effect, easy to use, low threshold, and short training time. developers can master the software for testing within 2 hours. Purifyplus consists of three products: Rational purify, quanify, and purecoverage. The main functions are as follows:

• Rational purify mainly targets memory errors and runtime errors that are difficult to find during software development. In the software development process:
  • Automatic Error detection;
  • Accurately locate errors;
  • Provides complete error information;

  This reduces debugging time, helps the development team identify defects, and ultimately forms high-quality products, so that you can truly release better software faster.

• Rational quantify mainly solves performance issues during software development. In the software development process:
  • It is easy to identify and display application performance bottlenecks to ensure the quality and performance of the entire application.
  • Rational quantify provides the development team with a global graphical view of performance data, allowing you to focus on performance issues from the beginning of the development process, so that you can release better software faster.
• Rational purecoverage provides Test Coverage Information for applications. In the software development process:
  • Automatically identifies untested code in the Code to ensure code test coverage;
  • Help developers ensure the quality of development and enable quality assurance personnel to evaluate the effectiveness of testing.
  • You can generate a comprehensive coverage report for each test. You can combine the overwrite data generated by multiple running programs and automatically compare the test results to evaluate the Test Progress.

4.5 automated testing of real-time system software using IBM Rational Software testing tools

IBM Rational test realtime is mainly suitable for developing software for real-time systems and non-real-time systems with high requirements. It can help the test team quickly establish testing capabilities such as unit testing, integration testing, and system testing. It provides important features such as automatic testing (including unit testing, integration testing, and system testing), code coverage, memory leakage check, performance analysis, and UML tracking, helps the software testing team detect and fix software defects before a system crash. Its main functions and features are as follows:

• Automatic Generation of test script templates and test programs (including the driver module and the pile module): Through source code analysis, the required test scripts and test programs run on the target are automatically generated. In addition to specifying test data using test scripts, manual encoding is not required. In the test report, the test results are associated with the source code to simplify code modification;
• Analyze code coverage, memory leakage, and performance bottlenecks through automatic code plug-ins, and establish associations with test cases;
• By associating the test results with the observed results with the tested code, we take the test as an important part of the development, and truly implement the development-side test and test-side observation, observe and evaluate the integrated development and testing process;
• Target deployment port (TDP), a common, low-cost, and easy-to-transplant target adaptation technology, is used to make tests irrelevant to compilers, connectors, debuggers, and target structures, implements a multi-target structure across multiple development environments;
• Model observation and code coverage integration: Use the UML trace function to observe the application running status, and use the state machine model coverage to associate test cases with models, make full use of the strengths of model and code-level testing;
• Integration with clearcase, ClearQuest, and RUP: completes Version Control of test files in the same integration environment, and submits and modifies change requests;

Back to Top

5. Summary

Figure 8. software automation test solution of IBM Rational

IBM Rational provides testing success experience, automated testing tools, and comprehensive consulting services for the software testing team, as shown in figure 8. The final implementation is: a testing team, based on a complete set of software testing procedures, using a complete set of automated software testing tools to complete a full range of software quality verification, this is the essence and ultimate goal of IBM Rational testing solutions.

Back to Top

Back to Top

About the author

		IBM has authored this article