Methods and practices of agile testing

Wen/Zhu Shaomin

Once, after the developer completes the code for the current sprint task, the testers, developers, and product managers come together to browse the product and go through it, and the product manager discovers the problem and thinks it needs to make a big change to the functionality. At this time the developer estimated that it would take two days to complete the code, but the testers opposed it, we had only 5 days to test time, plus this new function is more, the development code quality is not high, acceptance testing has been very tense. If it is delayed for another two days, the test cannot be completed. Product manager said, you are not using agile testing methods, should be measured quickly, three days should be able to complete the test work Ah!

What is agile testing? Agile testing, of course, cannot be simply understood to be faster, and is definitely not a test that takes less time than before, or reduces the scope of the test or reduces the quality of the test task. Others say that only agile development, no agile testing. Here's what we're going to discuss:

• What exactly is agile testing?
• What are the process improvements in agile testing?
• How do testers face the challenges of agile testing?
• How to develop the corresponding automated testing strategy in agile testing?

What is agile testing

If the past traditional testing processes and methods are stuck into the agile development process, the test work may be inefficient, the tester may be working overtime every day, and not play its due role. Agile testing should be a new test process, methodology, and practice for adapting to agile methodologies, tailoring the traditional testing process, focusing on different aspects, such as reducing the proportion of test plans, test case designs, and increasing communication and collaboration with product designers and developers. In the agile testing process, participate in unit testing, focus on new features of continuous iterations, perform adequate acceptance testing for these new features, and regression testing of legacy functionality relies on automated testing. Due to the short iteration cycle in agile methodologies, testers start testing early, including timely review of requirements, development design, and, more importantly, timely and continuous feedback on the quality of software products. In short, agile testing is the ongoing response to software quality issues in a timely manner, as shown in 1.

Figure 1 Image description of the Agile test definition

Optimization of the agile testing process

In the agile approach, the demand changes faster, the product development cycle is very short, we currently take four weeks, that is, a new version released every month. Short development cycle, continuous accumulation of functions, software testing brings great challenges, software testing process to do the corresponding adjustment. For example, our original test specification clearly stipulates that the main test plan for the project is established, then the test plan for each functional task is established, the test plan has a strict template, and it needs to be discussed with the product manager, the developer, and other people in the Test team, including the Test manager, The final approval and signature will be passed, and only the test plan will take one months to "draft, review and issue" a complete cycle. In an agile approach, it is no longer required to write a dozens of-page test plan, but to write a one-page test plan for each iteration cycle, listing the test points (including strategies, specific methods, focus areas, etc.).

In the original test specification, it is required to use Excel to write test cases, then discuss, review, review and then import the test Case library (online management system). In an agile test, you might not need a test case, but you can validate it directly against use cases or user stories and perform exploratory testing. and save time, for the development of the original function of the automated test script, for regression testing services. Automated test scripts will replace test cases and become a wealth of software organizations. The original test specification also requires two rounds of regression testing, in the agile test, only one round of regression testing. In combination with these considerations, the agile testing process is simple and effective, as shown in 2.

Figure 2 The Agile test process brief diagram

In the agile testing process, as mentioned earlier, testing is an ongoing quality feedback process, and the issues identified in the test are fed back to the product manager and developers, and some key areas are paid enough attention to, mainly:

• Testers not only need to participate in the whole process of demand, product function design and other discussions, but also to face to full discussion (including the language, video, instant messaging), simply by mail is not enough.
• Participate in code Review and properly assist the developer in unit testing.
• Add a link to the process "product walk-through"-testers and product managers, developers, and so on, to see the new features from start to finish, and to identify problems visually and quickly.

Testing and regression testing strategies for new features

Test tasks can be easily divided into new functional tests and regression tests. In agile methods, the corresponding strategies are established for testing the two parts to improve the efficiency of testing and to minimize the quality risk. The strategies for new functional testing include:

• Test cases are not required, and the validation of new features is done directly based on use cases and understanding of requirements. Even if you want to write test cases, just make sure that each function point is overwritten, not too detailed (large granularity).
• Validation continues, and once a new piece of code is completed (code Drop), it starts the validation instead of waiting until all the code is complete before starting the test. This also includes participation in unit testing and integration testing.
• Implement end-to-end (end-to-end) testing to ensure complete business process implementation, but also easy to find the business logic is not clear enough, unreasonable and other aspects of the problem.
• Read the code to find the problem, keep pace with the developers, and eliminate the stress of the test cycle.
• Based on experience, more exploratory testing, combined interaction (interoperation) testing, and user Scenario testing can be implemented to more effectively uncover deeper buried defects.

Regression testing is a difficult problem to be faced in agile testing. Each iteration adds new functionality, and a product may go through more than 10 or even dozens of iterations, the regression test range is increasing, and each iteration period remains the same, possibly one months. This is a very limited time for acceptance testing, so regression testing relies heavily on automated testing because it is difficult to control regression testing within a very limited range. Of course, there are ways to help us reduce the scope of regression testing, such as:

• By executing code Diff to understand all the places where the code is changing, and then by doing a correlation analysis, you can clearly know where regression testing is going, and the regression test scope will be greatly reduced.
• Reducing the scope of regression testing based on risk and operational surface analysis, such as regression testing, is just a matter of ensuring that the main function points are not a problem, but ignoring some of the details.
• The process of continuous testing, as long as there is time, to test, including developers, product designers are involved in the daily trial and test.

Automated test Strategy

Due to the short development cycle, requirements, design and other aspects of communication also takes a lot of time, there is not enough time to develop automated test scripts, at least the testing of new features difficult to achieve automated testing. At this point, the right strategy is needed to improve the effectiveness of automated testing, as shown in 3, and described below.

Figure 3 Strategies for automated testing

• Building a flexible and open automated testing framework, such as a keyword-driven automation framework, makes test scripts easier to develop, and facilitates script maintenance.
• Develop automated test scripts for stable product features, i.e. scripts to develop automated tests for pre-completed functions, while most new functional tests are performed by manual testing
• Focus on automating testing at the unit level, primarily by developers, testers who provide a unit test framework, and assist with some of the basic work required.
• The requirements of automated testing are well considered in product design and programming, making it possible to complete, automated, bottom-level testing, interface testing, and to avoid automated testing of the user interface (UI) as much as possible.
• Good IT infrastructure, including automated build packages, automated version verification (BVT), automated deployment, automatic coverage generation, and more.

Agile testing Tools

Automated testing relies on testing tools, but fortunately, there are many agile testing tools available. Due to space constraints, this is simply a list of common agile testing tools that are not discussed in depth.

• Unit Testing tools: TestNG, Xunit family (e.g. JUnit, NUnit), JMock, Bizmock, etc.
• Functional Test Automation: ThoughtWorks Twist.
• Web Functional Test (frontend): Selenium ide/rc, Watir, WatiN.
• WEB Service Test Tool (Backend): SoapUI.
• Performance test: Jmeter+badboy.
• Acceptance Test framework: FitNesse, Tellurium.
• Agile Test Process Management tools: Microsoft Visual Studio 2010, including TFS 2010, Scrum templates (MS VS Scrum 1.0), Test Manager 2010, Coded UI test, and more.
• Test framework for Business Intelligence (BI) applications: Oraylis BI. Quality (+ NUnit).
• Other collaboration tools, such as Testlink, BugZilla, bugfree, wiki, etc.

The value of testers in agile methodologies

In the agile approach, the leading role of the developer is more obvious, the system design, programming implementation, unit testing, refactoring and other seemingly critical tasks are falling on the developers, testers easily marginalized. So how is the value of testers reflected in agile methodologies?

• In the requirements and functional design discussion, testers can stand on the customer's point of view, play "User representative" role, emphasizing the user experience, truly reflects the complementary role of testers and developers.
• Testers not only play the role of "user representative", but also provide quality feedback in a timely manner through requirements discussions, code review and other activities, including code quality, interface consistency and so on, to ensure that quality is paid attention to throughout the process of product construction to improve the continuity and visibility of quality improvement.
• Testers should be actively involved in unit testing, and even if they do not participate in unit testing, they should be urged to conduct unit tests to ensure that unit tests reach more than 80% coverage, ensuring that the code is developed with good testability.
• In agile methods, it is often possible to break down a large system development into smaller subsystems (modules or components), integration testing and end-to-end (end-to-end) testing are more important, and testers can play a greater role in these tests.
• Before the release of the product, acceptance testing and regression testing are still indispensable, and this is where testers come in.
• At the end of an iterative cycle, the root causes of defects are analyzed and summarized to help developers establish good habits, prevent defects and fundamentally improve product quality.

Ideally, the tester has a design pattern, good programming skills, and a role swap with the developer, such as a tester role in the current version development, and a developer role in the next version development. In this way, the two sides have a deeper understanding of the work of different roles, eliminate the barriers of communication, and further improve the efficiency and quality of development.

Summarize

Based on the above discussion and our practice, a simple summary of agile testing is:

• Agile testing is the continuous testing, continuous feedback, playing "user representative" role, to ensure that products meet customer needs.
• Agile Functional Testing = Manual testing of new features (use case validation and exploratory testing) + automated testing of legacy functionality (regression testing).
• The difference between agile testers and developers is getting smaller, and ideally, in agile methodologies, testers and developers can be interchangeable in different iterations.
• Agile testing process According to different team characteristics, different product characteristics and different, local conditions, suitable is the best.

Author Zhu Shaomin, senior QA director, WebEx (China) Software Co., Ltd. Member of the Teaching Steering Committee of Software College of China University of Science and Technology, senior expert of China Software Testing Certification Committee (CSTQB). In the field of software engineering has been a lot of achievements, has won a number of scientific and Technological progress awards, published more than 10 books and university quality textbooks, such as "Full software Testing", "software testing methods and technology" and so on.

Methods and practices of agile testing