"Reprint" Regression test: Manual testing or Automation?

 

The problem that arises at every stage of a program's development is that repeated regression testing takes too long and often proves that the function does not change significantly. Thus, the likelihood of defects being present is quite low. Naturally, the result is that we come up with an idea to automate all regression testing scenarios and reject manual regression testing. The benefits here are very obvious:

• Automated Testing faster than Labor ;
• automated testing can be done at any time ;
• automated testing is very accurate ;
• Automation can be used in almost all testing processes
• Reports can be created automatically.
• Unfortunately, there are few discussions about the pitfalls contained in regression test automation.

Before you begin automated regression testing, there are several issues that you must address:

• what features will be covered by the test ;
• Automatic test architecture;
• Select the tools used for regression test automation;

we go back to the actual practice. In the step from manual testing, I have a project in the automated test switching phase, I have to test two types of account registrations in 43 domains determined by the country in which they are located. It's a terrible thing. The customer is not concerned about the quality of the information displayed.

Requirements are as follows:

• There are no fatal errors when creating an account ;
• The input information is consistent with the values that appear in the settings after creation.

After several runs, I realized that I had had enough. At that moment, I only saw that the number of fields registered varies by country, and I have some coding experience in C #. The next regression test is at your fingertips, and I can't hesitate. I asked my friends who were familiar with automated tests how to write Tests. After a lot of problems, attempts and errors, a simple location test was born. I found the locator and selenium Web driver--you see, all domains are using the same locator. I have a little thing to do--run the last manual regression test for all domains in life, and fill in the fields in the table and the corresponding fields. A new round of manual regression--a long process and complete the form--looks like it's all done.

I like the return of every one. I can watch some TV dramas while running the regression test. Incidentally, this coincides with a set of time. Although I have spent about 4-5 hours testing 43 domain registrations, now automated regression tests only take me 40-45 minutes to provide non-optimized code, no reporting system, no proper exception handlers.

according to the above, we can get Conclusion #1 : to say the advantages of automated testing, we should start with the most repeated tests, which are usually positive, but they need to run after each version.

This situation lasted several versions until the company decided to formally automate most of the functionality of the tests. Since our customers are not proficient in automated testing, we have decided to automate all of the areas within my knowledge of automated testing.

As a result, our automated testing has covered the entire system. Everything should be all right, but some of the revised tests start failing. In most cases, this occurs because of a long script that contains many steps:

• the test can fail in the second step, as the first part of the test, often bring the system to a certain state, running the whole test is meaningless ;
• lengthy scenes contain a lot of code and similar actions , which makes many of the tests that repeat the first step can cause fail.
• because of the repetition of the code, the automated test checks the same parts many times, which leads to an increase in unnecessary test time.

on this basis, we can get Conclusion #2 : long scenes should be run manually because it takes a lot of time to write them, and they tend to fail because of a few minor errors or mistakes that the second step makes.

Let's take a look at another very interesting situation. The customer orders the same automated tests as usual, and asks to write a few examples. He approved the idea, and within a few days we added a plan for automated testing. However, our programmers did not catch the deadline and finally had to write automated tests based on the current situation. More than half of the automated tests were unqualified as programmers delayed their release for a day or two.

Conclusion 3: never start writing automated tests before you have a stable version.

Just a moment ago, we have a more funny thing. We received a command to complete the automated testing of a website and have done everything quickly. Everything is more smooth than ever. A few weeks later, the client sent a letter complaining that the test could not be run. It turns out that he started to redesign the site, so all the tests failed. The matter taught me the importance of testing stability.

therefore, Conclusion #4 : If possible, use the locator, so that the element can be found after a minor change.

Therefore, the advantage of manual testing is clearly seen -it does not depend on anything that can be done at all times. There's no benefit in abandoning manual testing. Automatic and manual testing are interrelated, complementary testing methods, each with advantages and disadvantages. Think about the time spent writing tests and support for a regression scenario before you automate it. Also, be aware that manual test specialists usually pay less than those who have the skills to test automation.

So, as always, the whole problem is actually the problem of money. If the regression test automation program saves money without sacrificing the quality of the product, you can expand the scope of the automation program until the product quality increases and you save the cost. The balance here is determined by you, depending on your experience and preferences.

Testbird Automated regression test platform for hand-tour /app developers to provide app Automation regression testing, simple click to automatically generate picture use cases, multiple phones simultaneously perform use case regression, baseline comparison, identify problems, adjust baselines, maintain test cases, and generate reports with one click .

"English Original:http://blog.bughuntress.com/automated-testing/ Regression-testing-manual-or-automated-with-examples"

"Reprint" Regression test: Manual testing or Automation?