Introduction to frontend Automated Testing
Background
Testing is an indispensable part of a sound R & D system. The front-end also needs to be tested. Your css changes may cause page dislocation and js changes may cause abnormal functions. As the front-end is biased towards the particularity of GUI software, despite the endless emergence of tools in the testing field, the automated testing at the front end is not widely implemented, and many people still focus on manual testing. This article attempts to explore tools and practices in the field of front-end automated testing.
Why automated testing?
A project will eventually go through rapid iteration to the maintenance-oriented State. introducing automated testing at a reasonable time can effectively reduce manual maintenance costs. The benefits of automated testing can be summarized as follows:
Automation benefits = iterations * full manual execution cost-first automation cost-maintenance frequency * Maintenance Cost
For automated testing, it is more inclined to avoid possible problems than to discover unknown problems.
Test direction
First, this article will not discuss the unit test direction, because the single test has a complete tool system. However, in front-end development, there are few frameworks and libraries that are willing to write a single test. In addition, the single test and maintenance costs are high, and it cannot meet all the requirements of the front-end test.
Frontend automated testing can be attempted in several ways:
Whether the interface regression test interface is normal is the most basic part of the front-end test.
Function Testing: whether the function operation is normal. Due to interaction, this part of testing is more complex than interface testing.
The performance of the performance test page is getting more and more attention, and the performance needs to be continuously concerned during the development process. Otherwise, it is easy to decrease with business iteration.
Page Feature Detection some dynamic areas cannot be tested through the interface comparison, and there is no functional exception, but may not meet the requirements. For example, the performance test on the mobile terminal shows that the big picture material detection is a feature detection. In addition, it is common to see whether static Resources in the page block meet expectations.
Open-source tools in the industry
To do well, you must first sharpen your tools. There are already many excellent frameworks and libraries in the field of automated testing in the industry, and they are good at using them to get twice the result with half the effort.
Interface regression test
Common Methods of interface regression testing are pixel comparison and dom structure comparison.
Pixel comparison
The basic idea of pixel comparison is that if the website is not out of order due to your changes, the testing page should be consistent with the normal page. It can be compared with the normal online page or the page history. Pixel comparison intuitively shows the differences in images. If a threshold is reached, the page may be abnormal.
Phanw.ss
Phanw.ss is a famous tool for Pixel comparison. Phanw.ss combines Casperjs and ResembleJs image comparison and analysis. It is good in terms of ease of use and comparison.
Does not support PhantomJS 2.0
Because PhantomJS 2.0 temporarily disables file upload, PhantomJS 2.0 is not supported by default. If you still want to use the method of modifying the source code to retrieve image files, you can use ajax to retrieve files under the same domain name. For details, refer to the ResembleJs official website example.
How to test multiple browsers
To test the compatibility of multiple browsers, you only need to obtain the compatibility of multiple browsers. Selenium is the most famous for multi-browser testing. selenium can automatically obtain multiple browsers. Front-end engineers can also use Node's webdriver to easily develop test scripts.
However, selenium requires a little higher installation and installation cost. In addition, for multiple browsers, the compatibility between different browsers is prone to errors. Different browsers may cause screenshot comparison failure due to a pixel deviation. multiple browsers may be more suitable for regression testing.
Responsive Page Test
Some foreign users apply the pixel comparison to the response page. If you use the same webpage for PCs and mobile devices, responsive testing can quickly return to your page on different sizes. Different from the response for mobile development alone, pages that support PC mobile are more prone to disorder.
For example, the BackstopJS project uses tools such as PhantomJS and capserJS to determine whether the BackstopJS project is normal in different sizes Based on the resemberJS pixel comparison:
Notes for Pixel comparison
We do not recommend that you test all pages of the website. This will only lead to alarms, but not necessarily errors. Testing for reusable components and styles and areas that are prone to problems is more effective
We recommend that you test the area instead of the entire page. Any dynamic changes such as text and images may fail, and the real error may be ignored because the image is too large. The larger the image, the easier it is to time out.
Hide a dynamic area. If a dynamic element exists in the area corresponding to the selector, you can also use the selector to hide the dynamic area.
Interface comparison is only one step. It is necessary to combine it with other tests without silver bullets. rational integration is the key.
Dom structure comparison
Although the pixel comparison is intuitive, there are many dynamic elements that cannot guarantee limited synchronization between the test page and online page. @ Yunlong daniu provides a new solution page-monitor for this problem. Compare the changes of the whole page based on the comparison of dom structure and style. Example:
With page-monitor, you can quickly build a monitoring system to monitor text and style changes on the page.
Pixel comparison and dom structure comparison have their own advantages, but they cannot solve all the problems. Why not make full use of it? QA colleagues in FEX provided the pagediff platform in two ways and are in Public Beta! If you are interested, try it ~ Http://pagediff.baidu.com
The platforms developed by QA personnel are so dazzling. How can the front-end not understand any testing knowledge?
User Operation Test
As mentioned above, functional tests cannot be replaced by interface regression tests. Even if the interface is normal, the function is also a must-note part. The most direct function test is to simulate user operations and determine whether the page display meets expectations by simulating the normal operation process.
Phantomjs and CasperJS
The well-known PhantomJS should be introduced to you! The previous interface comparison tests are basically developed based on PhantomJS. Phantom JS is a WebKit for JavaScript APIs on the server side. It supports various Web standards: DOM processing, CSS selector, JSON, Canvas, and SVG. We are confident in web Testing, interface, network capture, automatic page access, and so on.
Casperjs is an encapsulation of PhantomJS. It provides easier-to-use APIs and enhanced support for testing. For example, with CasperJS, you can easily implement the auto-posting function of the post bar:
Casper. test. begin ('test posting function', function suite (test ){
// Log on to Baidu
Casper. loginBaidu (); // implementation is omitted, which can be achieved through cookie or form Login
Casper. thenOpen ('HTTP: // tieba.baidu.com/p/3817915520', function (){
Var text = "good guy ";
// Wait for the post box to appear
This. waitForSelector (
'# Ueditor_replace ',
Function (){
// Start posting
This. echo ("start posting. Post content: "+ text," INFO ");
// Execute js
This. page. evaluate (function (text ){
$ ("# Ueditor_replace"). text (text );
$ ("A. poster_submit"). click (); // click to submit
}, Text );
}, Function (){
Test. fail ("the Posting Box cannot be found # ueditor_replace ");
}
);
})
. Run (function (){
Test. done ();
});
});
With the most familiar front-end language, you can easily implement the Automatic posting function with just dozens of lines of code. You can also add some test logic to improve the case.
Compared with a single test, casperjs can use simple APIs to quickly test whether the functions of the website are normal from the perspective of actual user operations, and retain the final operation flow visualization for each test step. For example, the GitHub project below uses Casperjs to test whether important processes such as logon and order placement of an e-commerce website are normal. After case is complete, a command can be used to test the entire website.
Casperjs can listen to the test and the status of the page for other operations. If the test results are slightly optimized, a visual operation flow can be formed:
This intuitively shows the operation status and the error steps (if any error image is red). The detailed log output of the casper test is shown below.
Do not want to maintain case?
Unless there are enough QA personnel to help you complete the test, manual regression will certainly consume more energy. In the stable phase of project functions, case maintenance is much simpler, and we also recommend that you add cases for website core functions instead of all functions.
Browser compatibility test
Of course, selenium also supports operation testing. Similar Tools include dalekjs. If you want to specifically test IE, you can consider [triflejs] http://triflejs.org/and refer to apiwith phantomjsbasic.
PhantomFlow operation comparison test
Is there a tool that is as intuitive as image comparison and easy to write case? You can consider PhantomFlow. If the page is normal, the test page and normal page should be displayed in the same operation. Based on this, you only need to define a series of operation processes and decision branches, and then use phanw.ss to compare them with images. Finally, it is displayed in a very good visual report. You can see the test visualization chart conducted by the author's company:
Different operations are represented in the image. Each operation has a decision branch, and each green point indicates that the image is compared normally. If it is red, it indicates an exception. Click here to view the operation details:
I have to say that this is a good idea. It condenses the case of the operation test into a decision tree. You only need to define the operation and perform the key part. If the website is static or ensures data consistency of the sandbox address, this test tool can effectively improve the efficiency of automated testing.
Performance Testing
Website display performance is also becoming more and more important. Especially, mobile performance is always an important factor affecting user experience. Generally, developers use automated tools to merge and compress resources and perform other optimizations. Many large companies also set up their own performance monitoring systems to guide optimization. For performance monitoring, refer to my other article to create a front-end performance monitoring system in seven days.
It should be noted that performance is not a goal, but a problem that requires continuous attention during development and testing. We have automated tools and frameworks optimized during development, and we can also use tools to perform performance tests during testing.
We recommend a Phantomas Tool Based on PhantomJS. It can run the test page to obtain many performance metrics, the loading time, number of page requests, resource size, whether to enable cache and Gzip, selector performance, dom structure, and many other indicators can be obtained at one time, and the corresponding grunt plug-in is available. You can also perform secondary development on the detection indicators. For example, you can define a rule for the maximum image size on the mobile end. When developing a large image that exceeds the limit, an alarm is triggered. However, if you use the time point during the loading process as the standard test and monitoring, it is best to simulate the mobile network environment.
Page Feature Detection and practices
As mentioned above, the size of test resources in performance testing is actually a kind of Resource feature. For example, we can develop some general test rules to test whether the page is normal. This test is mainly applicable to elements that cannot be directly judged on the interface and operations. For example, whether the advertisement deployment on the page is normal.
Advertisement deployment detection practices
Problems may occur when third parties deploy advertisements and configure materials, for example, code script upgrade error, deployment error, incorrect material size format, advertisement container not adapted to multiple screen sizes, whether the advertisement is visible, and whether the timeliness advertisement is displayed. There are many known problems. In the case of problems, it is very labor-consuming to detect them one by one. These features are related to the actual running environment. Most of them can be detected using tools such as casperjs.
In addition, there are also ad-related blocking checks to check whether the div ad block (non-iframe ads) on the page is intercepted by the blocking plug-in. Because the interception plug-in uses basically the same interception rules, and the div advertisement uses selector shielding, you only need to judge whether the selector exists on the page according to the relevant detection rules. This can be done with an api in casperjs:
If (casper. exist (selector )){
Casper. captureSelector (filename, selector );
}
In this way, you can directly intercept the area.
Integration with automated testing
Back to the previous requirements, how can we achieve these detection requirements through casperjs. Casperjs supports executing JS to obtain the returned results:
This. page. evaluate (function (text ){
$ ("# Ueditor_replace"). text (text );
$ ("A. poster_submit"). click (); // click to submit
}, Text );
In addition, you can actively inject jquery, zepto, and other frameworks, so that you can operate and analyze dom elements in a very simple way. For example, you can obtain the deployment type based on the html structure features, automatically scan the container width for AD detection, and obtain the ad selector for screenshots. If an error is reported on the page, you can use the casper api to listen:
Casper. on ("page. error", function (msg, trace ){
This. echo (msg, 'warning ');
// Detailed error message
If (trace ){
This. echo ("Error:" + msg, "ERROR ");
This. echo ("file:" + trace [0]. file, "WARNING ");
This. echo ("line:" + trace [0]. line, "WARNING ");
This. echo ("function:" + trace [0] ["function"], "WARNING ");
}
});
It can also capture network request analysis dead links or ad requests:
// Record all requests
Casper. on ('Resource. requested', function (req, networkRequest ){
// Do something
});
Even better, you can go to the iframe for cross-origin operations!
Casper. withFrame (id/name, function (){
// Now you are inside iframe
})
Note: name is recommended for iframe operations, and the id is sometimes misplaced.
It can be said that with such a good tool, you can easily achieve many unexpected needs!
Reduce configuration costs
After the detection tool is developed, we certainly need to reduce the cost of use. For example, in the above example, we only need to configure some rules and detection pages for advertisement detection, users only need to pay attention to the pages to be tested. The tool automatically runs based on the configuration submitted by the user and returns the result to the user.
Integration with CI
At this point, the above steps are very similar to the ci system! If ci can be used to implement a series of automated deployment and testing tasks, it will be easier to use.
Talking About ci, we must introduce jenkins. It is stable and reliable, and is the first choice for many large companies. It just looks like it in the eyes of the front end .. Ugly and difficult to use .. How nice it is to be as fresh and intuitive and easy to use as travis-ci.
Of course, if you want to implement a ci-like process by yourself, for the automated test mentioned above, only one queue system needs to process the batch submitted test tasks and then feedback the running results to the user. Of course, front-end testing may require higher output requirements for Custom reports. If you want to implement one set, laravel and beanstalkd can be used to quickly build a complete queue system. laravel already provides many built-in support. A lightweight ci system that supports custom display can be implemented by outputting the running results of each service into an html report. There are many tutorials in this regard and you can search for them by yourself.
Good lightweight ci systems made abroad include:
Http://wercker.com/
Https://semaphoreci.com/
Https://codeship.com
Https://circleci.com/
...
Good user experience makes people feel happy and there is no obstacle. If you want to customize it, you can use it as a reference.
Practical experience
Front-end automated testing can be said to be a field that is constantly being explored, and problems are inevitable during implementation. Some important points can be used as experience reference.
Reduces use and maintenance costs
The disadvantages of automated testing are nothing more than the use effect and cost. You can select the right tool for the use of results, while the use cost can be reduced to a reasonable degree through a series of measures:
Combined with the building tool grunt and fiis, automated testing and building tools can be combined to detect problems earlier, which can also reduce the use and maintenance costs.
The combination of CI system and continuous Foundation gives full play to the role of Automated Testing in a wider scope.
The combination of workflows and daily workflows also aims to reduce the cost of use, for example, feedback the results to users through custom methods.
Test configuration-based test configuration makes it easier for users to use and maintain. In most cases, you only need to maintain the configuration script.
Focus on details to improve problem locating capability
Each product has its own characteristics. If you just use these open-source tools in a rough way, you may not be able to achieve the desired results. You need to select a reasonable tool based on your own situation and make certain optimizations. Only by constantly improving the problem locating capability of automated testing can we truly realize the value of automation.
Utilize open source power and use it properly
If you have any problems, find a solution.
Find open-source support based on your ideas
If not, refer to article 1.
The open-source world already has many excellent resources. We do not recommend you start from scratch unless you can maintain them well. Optimization and secondary development based on existing excellent tools, libraries, and platforms are more conducive to the development of tools.
Summary
Testing is an important part of R & D. Although front-end automated testing is still being explored, there are already many excellent tools and libraries.
Reasonable Use of tools, targeted selection, and reduced use and maintenance costs.
