Automated test: Is it really silver bullet? _ Automated Testing

Turn from: https://www.ibm.com/developerworks/cn/rational/r-testing-automation/

Brief introduction

Frederick p. Brooks, Jr. In 1986, wrote an article entitled "No Silver Bullet: the fundamental and secondary issues of software Engineering" (No Silver bullet-essence and accidents of Software Engineer ing). This article lists some of the expectations of the development of software engineering technology and compares it with reality. His arguments were summed up as follows:

Without a single technical or managerial advance, the ability to independently commit to significantly improve software productivity, reliability, and simplicity over a period of 10 years

Brooks encourages us to view technology and methods as an evolutionary tool, not a revolution. I tend to support the same point of view when introducing automation technology into the test work.

I've been dealing with potential customers of automated test products and solutions for 5 years, and I've met a lot of "silver bullet" ways of thinking. They always come up with ideas like this: All tests can be automated. Since automated testing can increase productivity so dramatically, we can do all the testing with fewer people (lean people). Automated testing is so simple that we don't need any training. Automated methods reduce the overall test effort. We do not need to develop any test plan. With automated testing, testers are not "obsolete" or "redundant". That time-consuming test design work is no longer necessary.

Although I do not want to break the illusion of good people, but always feel the responsibility to help them understand, the implementation of automated testing and the dream of the divine weapon of the difference between. This usually means interpreting the true meaning of automated testing and the actual functionality of automated test tools and solutions.

The automated tests are not silver bullets.

That's what it means. Automated testing, or the implementation of automated testing strategies and tools, is just one tool in the tester's toolbox. Note that I emphasize that it is a tool in the toolbox. I deliberately avoided equating the automated test with the tester, which could not have replaced the status of testers. However, automated testing remains unquestionably powerful and can benefit us in terms of testing efficiency and thoroughness. The key is to determine the best time and manner to perform its function. Let's ask another question to elaborate.

Do you have enough time to test every thing?

I think people will answer in unison, "No. "。 There's always more to be tested, or try again on another platform or in another configuration. But with deadlines and product delivery dates looming, the time allotted to each test cycle is shortened. So how does the software development project manager and the Test team deal with this situation? Typically, they reduce the amount of testing per test cycle before the software is released. Have you ever experienced this situation? Ideally, you need to do some risk-based analysis to decide which risks to queue. More often than not, however, the Test team simply concentrates the entire test cycle on verifying the bugs that have been repaired. Even this shrinking test plan does not have enough time to complete.

How many products have been delivered after the full test. I don't know much about this situation. The development team often makes decisions about whether to deliver the software based on other factors: is time up? Is the budget too large? Have you exhausted your resources? Do you have any pizza or beer?

Unfortunately, as test work is arbitrarily truncated, the development team is not fully aware of the overall quality of the product, and they face the risk of serious problems with the software being delivered. With the power of automated testing, can we get out of this mess? Let's move on.

How automated testing can help us.

Before you plan to implement automated testing, you need to understand the definition of automated testing. In other words, what it means to you. Here are some of the other people I've heard about automated testing: completely unattended testing. Test the script. Test tools. I don't know.

Sometimes people take the concept of automated testing too narrowly and care only about test scripts generated by tools or programming. In fact, the word automation contains a broader meaning. Look at a quality engineering team's definition of automated testing when building a set of automated test guidelines:

In our environment, "automation" refers to the use of policies, tools, and artifacts, which increases or reduces the need for manual or human involvement or intervention that is not skilled, repetitive, or lengthy.

In addition to this definition, the guidelines provide an example of how automated methods are applied to the team. Table 1 lists a few.

This small example lets you look at automation from another angle. Now, defining what automation means to you and your team is critical. You can then use this definition to start building a set of automation guidelines so that everyone on your team can use the same approach and quickly assess whether a task is appropriate for application automation.

Creating Automated Test Guidelines

Here are some of the strategies and things you can consider when you define automation and develop guidelines:

Identifying the "where" of automated tests will be a candidate for application automation for a specific part of all work. Start with a highly redundant task or scenario. Automate tedious and artificially error-prone work. Focus first on developing a mature, thoroughly understood use case or scenario. Select the relatively stable part of the application, rather than the variable part. Improve automation (increase the depth and breadth of test coverage) by using data-driven test techniques. Assign several experts to automate and don't let everyone in the Test team do the work. Keep in mind that not to pursue 100% of automation, manual testing remains critical.

More tests are planned to automate repetitive tests and to gain more time for other methods of testing. Add exploratory tests. Increase the configuration test. Build more automated tests. More manual testing, especially with regard to high risk characteristics. Careful planning: The Division of labor testing and automated testing will not be completely automated. Every design has to design all the tests and documents. If an automated test fails to work, make sure it is done manually.

Consider automation as an investment training user to take full advantage of automation tools. Build a reusable code base. Keep the test modular, size control within a certain range, so easy to maintain. Document the test script (code) for verification and reuse. Strengthen the backup process. Use source code control. Recognizing that automation is a software development effort, code generation is often required.

Step through the automated tests without trying to automate all the tests in one day. To accumulate experience and to be gradual. Start with a small portion of the entire test plan and incrementally add it to the automated test collection. (i.e. increment in actual, controlled manner)

What else can automation do for me.

While automated testing requires a huge investment in upfront planning and training, it does bring a few big gains. It brings you the benefits of higher quality software-because you can spend less time and resources doing more testing. The potential for more complete test coverage. Devote more time to other testing activities, including detailed planning. Carefully design the test. Build more complex tests (data driven, add code for conditional branching and special reports, etc.) more manual testing, not less.

Automated testing also provides you with intangible value, which gives testers the opportunity to acquire new skills (i.e. to build skills and learn skills). Learn more about the knowledge of the system in the test, because automation can reveal the internal state of the system, such as object properties and data. (More understanding of the system creates better testers)

Now that you know what an automated test is and what it is capable of, I hope you can use that knowledge to do more and better testing of your product. Although automated testing is not a silver bullet, it is still an excellent tool, and if you can apply it to the right job, it will bring you great benefits.