Automated Testing ROI Analysis (2)

1. Factors affecting the ROI

ROI is usually calculated based on the income gained divided by the input cost. If we start a new project, we useTestThe value is divided by the test cost to calculate the ROI of the test. Sometimes,Automated TestingThe introduction takes place after a period of time that the manual test has been completed.

The economic cost of automated testing can generally be described as fixed or variable costs. Fixed costs include equipment, tools, and training. The fixed cost is not affected by the quantity and number of running results of automated testing. The variable cost increases or decreases with the number of developed automated testing results and the number of automated testing operations.

Example of fixed costs of automated testing:

1) Hardware

2) application software license

3) technical support of Application Software

4) design and build an automated testing environment

5) maintenance of the automated test environment

6) script development tools and software

7) Script Development Tool license

8) Test Tool training

9) Introduction and start of testing tools

Examples of variable costs of automated testing:

1) Design of automated test cases

2) implementation of automated testing

3) maintenance of Automated Testing

4) automated test case execution

5) Analysis of automated test results

6) Report of defects

7) Test Result report

8) Saving test execution data

9) automatically executed tests

Manual and automated testing have some common elements.

Examples of common elements:

1) Analysis of tested software

2) Test Plan

3) Basic test design

4) Report of defects

5) Management of test result reports

When calculating the economic elements of automated testing, we can compare it with two things: manual testing or no testing (taking unknown risks without testing ).

When calculating the return value, we need to select the calculation time period (t ). Generally, you can determine the computing cycle based on the milestones of a project. In addition, the reward for automated testing is after the release of the new version. You can also determine the computing cycle based on the release of the new version. At the same time, it must be consistent with the next release and next release. Using these two computing cycles to calculate the return of automated testing can help us understand the benefits of long-term and short-term automated testing very clearly.

The fixed cost of automated testing is not the absolute value. These costs need to be allocated in stages within their useful life cycles and adjusted using time periods (t. T value should be selected based on management factors, such as the time interval between product releases, ROI calculation, expectations for tool service life, and expectations for testing life, in order to make the T value calculated reasonable, useful and simple. The cost is multiplied by T and divided by the service life. For example, if the price of a tool is 25000 Yuan and the expected time is two years, the cost for the first year is 12500 yuan (25000*1/2 ). If four years are used for calculation, the cost is 50000 yuan (25000*4/2 ). The cost of investment must be calculated within the service life of the tool. If the service life of the tool is one year, the cost for the first year is 25000 yuan. (Similarly, if a person who has received the training leaves his/her department after the training, the whole training cost will be lost, this cost cannot be apportioned within the time period ).

Compared with manual testing, the biggest value of automated testing lies in the low cost of each test run. This brings about two factors for calculating the ROI: the number of automated tests (N1) and the number of manual tests (N2.

Automated Testing requires maintenance. Therefore, the number of running times of automated testing scripts before the change is very important. Many automated tests are difficult to run because of frequent GUI changes. The automated test group uses the recording/Playback Technology to create an automated test script and measures the three times required for manual testing.WorkQuantity. Maintenance includes rerecording test scripts and test results during the synchronization of testing and software development. It seems that the automated test group does not perform much test, but keeps rerecording. Therefore, after recalculating the average running times of the automated test script (before the change), we found that this number is 1.2. 4/5 of scripts run only once (before you have to record them ). Finally, this low-productivity recording/playback method had to be abandoned.

 

For costs, these factors can be divided at a deeper level. One is the same nature between automated testing and manual testing, and the other is increasing or decreasing. These common influencing factors can be left out of automated testing ROI computing because they are neither cost nor benefit. When we perform automated testing, the increasing impact factors can be seen as costs, while the decreasing impact factors are seen as benefits. Some factors are always increasing or decreasing, and most of the changing factors can be costs or benefits, mainly depending on the type of automated testing and the results achieved by automated testing. The following are some examples:

Changing factors (can be the cost of automated testing, or the benefit ):

1) maintenance of the automated test environment (which may be an increasing cost, or may be continuously reduced throughout the maintenance cost)

2) Execution of Test Cases

3) analysis of test results

4) Report of defects

5) test result report

6) test data generation

Benefits of automated testing:

1) Save test execution

2) work after the test is completed automatically by the System

Cost of automated testing:

1) Hardware

2) software licenses in the testing environment

3) Technical support for software in the testing environment

4) Design of automated test environment

5) implementation of an automated testing environment

6) script tools

7) Test Tool license

8) Test Tool training

9) Introduction and start of testing tools

10) Design of automated test cases

11) implementation of automated test cases

12) Maintenance of Automated Testing

 

From: http://www.51testing.com/html/02/n-100502.html