Common pitfalls in Agile software testing

Transferred from ThoughtWorks

Agile Software Development is a kind of new software development method which has been paid more and more attention since the 1990 's, and it is a kind of software development ability that can cope with the fast changing demand, it is applied to the software project more and more as a new development mode.

Agile Software Testing refers to a series of activities related to quality in the process of agile software development, and the traditional sense of the software test there are many differences, because the concept of agile software testing has been relatively vague, so often people go into the wrong, I have been in the Waterfall software development model for several years of testing personnel, So there have been some misunderstandings when it comes to agile projects, but after nearly 5 years of working with the Agile Software Development team, there is a new understanding of a number of issues, and here are some common misconceptions to share with you.

No test strategy required

The test strategy is focused on goals and methods, that is, how to effectively use limited resources to achieve the goals set in advance in a defined time period, when the test strategy is developed, the test objectives are defined first, and then the test types are determined, the approximate proportions of the various test types are selected Finally, plan what test phases you need to go through before you release the software.

Many people think that agile software development is based on user stories, is it enough to focus on user story testing? Is there no need to consider the test strategy at all?

This is a big misunderstanding, because agile software development is usually an iterative release, cycle is relatively short, resources are very limited, which requires us to co-ordinate planning, small to a user story, large to a complete user characteristics, all need to consider how to reasonably use test resources, so agile project is very need to test strategy.

Specific to the actual project, typically, the team develops a test strategy at the beginning of the project (iteration 0), identifies the objectives (including the objectives of the functional requirements, and the objectives of the non-functional requirements), and then determines which automated tests to add in the development phase (including unit testing, interface testing, contract testing, integration testing, System-Level UI User scenario test), and specify the approximate ratios of these tests (conforming to the test pyramid), choosing an automated test framework (such as Xunit) and what manual tests are required (including exploratory testing, usability testing, and so on), and planning the testing phases that need to be performed for each release cycle (for example, new feature testing , regression testing, etc.), then the test strategy will play a very important role in the development and testing of agile teams, and of course, each team will be different because of the different strategies of the project.

The following figure is an introduction to a simple agile test strategy:

No test document required

A test document typically includes a test plan, a test case, a test report, a test defect, and a portion of the requirement document that corresponds to a guide to the test.

A lot of people would think that Agile software testing does not require documentation, and in the Agile Manifesto there is a "working software above exhaustive documentation", although the Agile manifesto concludes with "The right item is valuable, we value the left," but people tend to overlook the content of the right item, Leads to a complete denial of the role of test documents in many teams that are just beginning to implement agile development.

First of all, it is undeniable that in real agile projects, it is very rare in the traditional sense of formal specialized requirements and test documents, but this does not mean that the Agile project does not have documents, such as user stories is the carrier of the requirements, user stories in the acceptance criteria is a part of the Agile test document, In addition, many agile software projects are developed in a BDD way, using test cases in natural language descriptions that business people can understand, combining automation to form a requirement and testing document, and in response to the problem of fast document updates that are not timely due to changes in agile software testing, Many agile projects are using the living document.

Pure automated test or pure manual test

Some of the agile people think that the Agile software development release cycle is very short, testers do not have time to do manual testing, so should use pure automated testing.

There are also those who argue that agile development emphasizes rapid response changes, and that if the cost of input is automated, it will certainly lead to a waste of resources in the maintenance of automated testing, so a pure manual test should be used.

These are two extreme misconceptions, although the difficulties that these two viewpoints take into account do exist, because in the Agile software development process, iterations are usually shorter and do not set aside enough time for manual testing, so there must be enough automated testing to protect them.

However, because the test code itself may be flawed, and many parts of it are difficult to be covered by automated tests (such as interface testing, usability testing, exploratory testing, etc.), agile testing is also inseparable from manual testing.

As for concerns about the cost of automated test maintenance, agile projects also do have a lot of changes in the characteristics, but often change is relatively close to the user's part, so you should minimize the user-level dependence on the interface of automated testing, and add some more difficult to change the underlying unit test interface test.

Agile testing is recommended primarily for automated testing, supplemented by manual testing.

Agile QA = Agile Tester

In many of the teams that have just come in contact with agile practices, the understanding of agile QA is still in the tester stage, and it is believed that as long as the user story is developed, QA goes to a full-time test and finds that the defect is enough.

This is a big misunderstanding, first QA (Quality assurance/analyst), not simply the tester of meaning, through the name of the role we can see that Agile QA emphasizes quality assurance and analysis, not simply testing products.

In an actual project, agile QA usually begins to participate in the entire software development process from the requirements analysis phase, help the team to reach a consensus on quality through different stages and roles in the team, and prevent defects by identifying and validating at different stages, rather than waiting until software development is completed to discover defects. So for agile QA, the goal is to find as few defects as possible after the software development is completed, and for tester, the more defects are found, the better the work is.