How to implement test automation
In order to build high-quality software, you must continuously
run automated and manual tests throughout the delivery process.
The automatic test includes the following items:
unit test. This type of testing usually tests a method, class, or function alone, assuring developers that their code is working as expected. To ensure that the code can be tested and that the test is easy to maintain, write the test before writing the code. This method is also called test-driven development (TDD).
Acceptance testing: This type of testing usually tests the entire application to ensure that higher-level functions perform as expected, and regression errors are not introduced. For example, the acceptance test can check the business acceptable standards, the correctness of an API, and whether the original function is functioning correctly for a certain user case. Such tests should be written as part of the development process. No one can declare that their code has been "developed" unless it has passed the automatic acceptance test.
Automatic testing is suitable for continuous delivery deployment pipelines. In such pipelines, every change runs a build to create software packages, perform unit tests, and possibly other checks (such as static analysis). After these software packages successfully pass the first stage, more fully automated acceptance tests will be conducted on the software that is automatically deployed and running, and some non-functional tests may be performed. Any construction that passes the acceptance phase will usually be subject to manual exploration and usability testing. Finally, if no errors are found during these manual steps, the application is considered ready for release. In other words, whether to deploy this application to the production environment will be entirely determined by the business department.
The continuous deployment pipeline can ensure that developers quickly get feedback, shorten the preparation time from check-in to release, and reduce the error rate in the production environment. Because most of the developer’s code can be verified in minutes (rather than days or weeks), they can fix the bug as soon as they find it.
In the deployment pipeline mode, each change will create a candidate version, and the rapid feedback loop helps to find problems in the relevant process as early as possible. If the team still lacks confidence in the released package when the package reaches the end of the pipeline, or if the team finds a defect in the production environment, then the pipeline must be improved (tests may need to be added or updated).
Common mistakes
The developer did not participate in the test.
Developers must participate in the creation and maintenance of automated test suites. If other teams are responsible for test automation, the following two problems usually occur:
Test suites are often in a failed state. After changing the code, you may need to update the test. If the test automation is not the responsibility of the developer, the build pipeline will remain in a failed state until the responsible team fixes the related tests.
Developers often generate code that is difficult to test. Developers often solve the problems assigned to them without considering how to test the code. This may lead to poor test suite design, or cause the test automation team to refactor large amounts of code.
This does not mean that you should abandon the tester or QA team. Testers can view the system from a unique perspective because they understand how users interact with the system. The best practice is for developers to work with testers to help testers create and improve automated test suites. In this way, the developer can understand the tester's perspective, and the tester can understand the automation. Testers should also perform exploratory testing as part of the deployment pipeline.
The ratio of unit tests to acceptance tests is incorrect. A specific design goal of the automated test suite is to detect errors early. Therefore, you should run the faster unit tests before the slower acceptance tests, and you should run both tests before any manual tests.
You should find the error through the fastest test category as much as possible. If you find an error during the acceptance test or exploratory test, please add a unit test to ensure that the error can be found faster, earlier, and at a lower cost next time.
The appropriate test suite has not been carefully arranged. E.g:
If you have to change multiple unit tests every time you change the code, you may be overly dependent on simulation or the unit test suite may not be optimized.
The acceptance test suite should generally reflect the actual end-to-end journey of the user using the system, rather than a collection of automatic acceptance criteria. For more information on this, please watch the video "Setting a Foundation For Successful Test Automation" by Angie Jones (laying the foundation for successful test automation).
Well-factored test suite. If each change to the interface will cause multiple acceptance tests to fail, then use the page object pattern to separate your tests from the system under test.
Ways to improve automated testing
If you do not fully implement
test automation, you can first build a framework deployment pipeline. For example, create a unit test, an acceptance test, and an automated deployment script for establishing an exploratory test environment, and combine them. Then, with the improvement of products or services, gradually increase the scope of testing and expand the deployment pipeline.
If you are already improving a Brownfield system, please follow the instructions in this article, but do not stop halfway, trying to improve a comprehensive automated test suite. The correct approach is to write a small number of acceptance tests for high-value features first. Then, be sure to ask the developer to write unit tests and acceptance tests for any new features and any features you want to change. You can consider using TDD to improve the quality and maintainability of the main code and test code. Finally, make sure to write unit tests when acceptance tests fail in the future to find defects faster.
If the test suite is expensive and unreliable, it should be eliminated decisively. If a test suite contains only ten tests, but it is reliable, fast, and trustworthy, and another test suite contains hundreds of tests, but it is difficult to maintain and no one trusts, then the former is far superior to the latter.
Finally, please ensure that testers and
developers collaborate during the development process, and not start testing after "development is complete". The ideal approach is to pair developers and testers to work together on the construction, classification, and maintenance programming of acceptance tests. If this is not possible, try using a screen sharing tool.
