Fully tap the potential of Automated Testing

Logigear
White Paper 《
Achieving the full potential of test automation
This section summarizes some automated testing experience:

 

Software Test automation has the capability to decrease
The overall cost of testing and improve software quality, but most testing
Organizations have not been able to achieve the full potential of test
Automation. organization groups that implement test automation programs run into
Number of common pitfalls. These problems can lead to test automation plans
Being completely scrapped, with the tools purchased for test automation becoming
Expensive "shelfware". Often teams continue their automation effort, burdened
With huge costs in maintaining large suites of automatic test scripts that are
Of questionable value.

 

The benefits of software test automation

Benefits of Automated Testing

Most software development and testing organizations are
Well aware of the benefits of test automation. A quick glance at the web sites
Of any test automation tool vendor will point out a number of the key benefits
Of test automation. Some of these benefits include:

 

Reduced test execution time and cost

(Reduce test execution time and cost)

:
Automatic tests take less time to execute than manual tests, and can generally
Execute unattended. A tester must simply start the test, and then analyze
Results When the test is completed.

 

Increased test coverage on each testing cycle

(Improve the test coverage rate for each test cycle)

:
Automatic tests can allow testing teams to execute large volumes of tests
Against each build of their application, achieving a level of coverage that
Wocould not be possible with manual testing. This increased coverage can help
Teams uncover bugs in existing functionality much more quickly than through
Manual testing. test automation can allow teams to test more features in each
Cycle (breadth), and also to test features using more permutations of inputs
(Depth ).

 

Increased Value of manual testing effort

(Increase the value of manual testing)

: So long
As applications are meant for human end users, test automation will never
Entirely replace the need for human testers. No matter how sophisticated test
Automation tools become, they will never be as good as human testers at finding
Bugs in an application. Human testers will instantly notice subtle bugs that
Are almost never detected by test automation, special usability bugs.
Automatic Test tools cannot 'follow their instincts 'to uncover bugs using
Exploratory and ad hoc testing techniques. By freeing manual testers from
Having to execute repetitive, mundane tests, test automation enables them
Focus on using their creativity, knowledge, and instincts to discover important
Bugs.

 

 

 

Pitfalls: Why test automation projects fail
To achieve their potential

Why does the automated test project fail?

 

Despite the clear benefits of test automation, validation
Organizations are not able to build valid tive test automation programs. Test
Automation becomes a costly effort that finds few bugs and is of questionable
Value to the organization.

There are a number of reasons why test automation efforts
Are unproductive. Some of the most common include:

 

Poor quality of tests being automatic

Automated test cases with low quality

Mark fewster explains this problem very well:

"It doesn' t matter how clever you
Are at automating a test or how well you do it, if the test itself achieves
Nothing then all you end up with is a test that achieves nothing faster.
"
[Fewster, software test automation, I .1, (Addison Wesley, 1999)]

Organization organizations simply focus on taking existing test
Cases and converting them into automatic tests. There is a sense that if 100%
Of the manual test cases can be automated, then the test automation effort will
Be a success.

In trying to achieve this goal, organization organizations find
That they may have automation failed of their manual tests, but it has come at
Huge investment of time and money, and produces few bugs found. This can be due
The fact that a poor test is a poor test, whether it is executed manually or
Automatically.

 

Lack of good test automation framework and
Process

Lack of excellent automated testing frameworks and processes

Many teams acquire a test automation tool and begin
Automatic as keep test cases as possible, with little consideration of how
They can structure their automation in such a way that it is scalable and
Maintainable. little consideration is given to manage the test scripts and
Test results, creating reusable functions, separating data from tests, and
Other key issues which allow a test automation effort to grow successfully.
After some time, the team realizes that they have hundreds or thousands of test
Scripts, thousands of separate test result files, and the combined work
Maintaining the existing scripts while continuing to automate new ones requires
A larger and larger test automation team with higher and higher costs and no
Additional Benefit.

 

Inability to adapt to changes in the system
Under Test

When the tested software changes, automated testing cannot be well adapted (maintenance problems !)

As teams drive towards their goal of automation as scheduled
Existing test cases as possible, they often don't consider what will happen
The automatic tests when the application under test (AUT) under goes
Significant change.

Lacking a well conceived test automation framework that
Considers how to handle changes to the system under test, these teams often
Find that the majority of their test scripts need maintenance. The outdated
Scripts will usually result in skyrocketing numbers of false negatives, since
The scripts are no longer finding the behavior they are programmed to exact CT.

As the Team
Hurriedly works to update the test scripts to account for the changes, Project
Stakeholders begin to lose faith in the results of the test automation.
Often
Times the lack of perceived value in the test automation will result in
Describe to scrap the existing test automation effort and start from scratch,
Using a more intelligent approach that will produce incrementally better
Results.

 

 

 

Generations: test automation Evolution

Evolution of Automated Testing"

 

Software Test automation has evolved through several
Generations of tools and techniques:

Capture/playback tools

(Recording playback tool)

Record
Actions of a tester in a manual test, and allow tests to be run unattended
Upload hours each day, greatly increasing test productivity and eliminating
Mind-numbing repetition of manual testing. However, even small changes to
Software under test require that the test be recorded manually again. Therefore
This first generation of tools is not efficient or scalable.

 

Scripting

(Script Programming)

,

A form
Programming in computer programming ages specifically developed for Software Test
Automation, alleviates extends issues with capture/playback tools. However,
Developers of these scripts must be highly technical and specialized
Programmers who work in isolation from the testers actually swimming
Tests. In addition, scripts are best suited for GUI testing and don't lend
Themselves to embedded, batch, or other forms of systems. Finally, as changes
To the software under test require complex changes to the associated Automation
Scripts, maintenance of ever-larger libraries of automation scripts becomes
Overwhelming challenge.

 

Data-Driven Testing

(Data-driven testing)



Is often
Considered separately as an important development in test automation. This approach
Simply but powerfully separates the automation script from the data to be input
And expected back from the software under test. This allows the data to be
Prepared by testers without relying on Automation engineers, and vastly
Increases the possible variations and amounts of data that can be used in
Software testing. This breaking down of the problem into two pieces is very
Powerful. While this approach greatly extends the usefulness of scripted Test
Automation, the huge maintenance chores required of the Automation programming
Staff remain.

 

Keyword-based test automation

(Keyword-based testing automation)



Breaks
Work down even farther, in an advanced, structured and elegant approach. This
Reduces the cost and time of test design, automation, and execution by allowing
All members of a testing team to focus on what they do best. Using this method,
Non-technical testers and business analysts can develop executable Test
Automation using "keywords" that represent actions recognizable to end-users,
Such as "login", while automation engineers should te their energy to coding
Low-level steps that make up those actions, such as "click", "Find text box
In window B "," enter username ", etc. keyword-based test design can actually
Begin Based on clients developed by business analysts or the marketing
Department, before the final details of the aut are known. As the test
Automation process proceeds, bottlenecks are removed and the expensive time
Highly trained specified sionals is used specified tively.

 

The cost-benefits of the keyword method become even more
Apparent as the testing process continues. When the software under test
Undergoes changes, revisions to the test and to the automation scripts are
Necessary. Organizing Test Design and Test automation with the keyword
Framework eliminates time previusly allocated to maintaining large libraries
Of scripts and rewriting entire scripts anew after major changes to
Software under test. With the keyword method, the necessary changes are far
Fewer. Modify changes do not require new automation at all, and can be completed
By non-technical testers or business analysts. When required, changes
Automatic keywords can be completed by automation engineers without affecting
The rest of the test.