Automated script execution Failure reason analysis

After the automation scripts have been executed, each use case is divided into two passes or failures. There is nothing to say about the adoption of use cases, which are mainly about the use cases of failure.


A failed use case requires someone to see if it is a problem with scripting stability or a problem caused by a program update.


The problem of scripting stability is divided into: problems caused by configuration environment and problems caused by non-configuration environment.


For problems caused by the configuration environment, it is necessary to manually or automatically check the environment and configure the environment before performing automated tests. This is how to configure, to know beforehand, written in the configuration specification.
The configuration environment causes problems, including:
A, the configuration of the automated test script.
B, configure the test program. such as: whether to restore the initial settings, whether to delete some data.
C, configure the browser.
D. Configure programs related to the test program or programs that affect the stability of the script.


For the configuration of the environment, for each test system, write the "XX system Automation Script configuration Manual" to avoid making low-level configuration errors.


The problems caused by the non-configuration environment are divided into the following categories:
A, network delay, identify the problem of synchronization of objects.
B. The script failed because of unknown factors.
C, unknown causes the script to run the interrupt.
D, the automation script itself uses the unstable factor.
E, the inheritance of the script, the last script failed to cause the next script also failed.


Problems with network latency, which can be resolved by re-executing the script again at the time of the error or by ensuring that the network is healthy before the script executes.
In the above categories, the Class E is the most severe, so it is better to write scripts without relying on scripts.
In the above categories, the most difficult modifications are in Class B and Class C, but you can debug by repeatedly running this class of scripts that cause failures.




The problems caused by program updates are grouped into the following categories:
A, program updates, resulting in a large number of script failures. A large number of scripts fail for a number of reasons and are complicated: the entire page has changed, business logic has changed, control types have changed, and program modifications are the most frequently used controls. If the business logic changes, it is more laborious to change.
b, program updates, resulting in a small number of script failures. A small number of scripts fail, generally the main process has not changed, it is relatively easy to modify.


In order to optimize costs, the execution of automated use cases is usually performed at night.
For known program updates, maintenance is generally performed prior to automated test execution.
For those who do not know where the program updates are, they are generally maintained after automated test execution.
If, after execution, there are a number of scripting stability issues or problems caused by a large number of program updates, it is necessary to perform analysis maintenance immediately the next day so that the maintenance is performed again the night after.
If there is a small number of scripting stability issues or problems caused by a small number of program updates after execution, it is up to you to decide whether to maintain it immediately and whether it needs to be executed again.


For the current week's work, the ideal scenario is that the maintenance script takes up 1 days, the execution time adds up to 0.5 days, and 3.5 days for other work.