When some "Recording playback" tests are used, test failures often occur due to seemingly unimportant ones. After understanding the limitations of such testing automation, the white-box tests are the same, find out the most important problems:
1. Behavioral sensitivity
If the system behavior changes and the requirement changes, the test fails.
2. Interface sensitivity
It is not a good idea to test the business logic in SUT (system under test) through interfaces. Even small changes to the water heater will lead to test failure.
3. Data Sensitivity Test
Data is stable only when data is not sensitive.
4. context sensitive
It is a prerequisite, status, and so on, which can easily affect the test execution result.
Next, I will try my best to find a solution to these four problems. In my daily work, these problems have always affected my testing process.