Visual perception Testing

With the advent of the second spring of the Internet and the prevalence of Web2.0, Web application development has become the main force of the current software development. Now, whether it's enterprise-class applications, social apps or mobile apps, the web has become a standard configuration, and many organizations are gradually using their own enterprise-class local applications for Internet web. However, the Web interface layout test, multi-browser testing, Css/javsscript and so on have become the pain of the interface test, especially the large-scale Web application regression test is too large, which leads to regression testing can not be completed at all, So there are very few teams that can complete a full range of interface layout regression tests, especially for teams developed with agile processes.

And now people are getting more and more attention to user experience and continuous deployment, which makes Web application interface development and testing more difficult.

First look at why web interface development and testing is so difficult.

1.Web Interface Layout Regression test

For Web page interface layout testing is generally manual manually comparing the design and product interface. There are two problems with manual comparison test: A, slow, b, human uncertainty. For enterprise-class Web applications with a large number of complex interfaces, the number of regression tests for interface layouts is huge, coupled with these two issues, which lead to a long and costly interface layout regression test for such applications, so many agile-based projects are unlikely to perform their interface regression tests at high quality in an iterative cycle. For a return every day, it is impossible to complete the task.

(There is a game "everyone to find fault", please look for the reader to find out how many differences, and record how much time spent. The answer is in the attached 1 figure)

Figure 1 People come to fault

2.css/javascript Code Refactoring

Now that the Web front end is getting more and more complex, the amount of code is also rapidly increasing, leading to front-end development starting with the development of library, module and pattern, just like back-end development. This creates a problem: how to quickly discover the side effect of the interface after the public code has been modified and reconstructed?

Since the JavaScript code of the CSS and control interface is refactored, it is only possible to manually detect its correctness through the human eye, which makes it difficult for developers and testers to check the interface that is affected by the modified code for a limited time. In the end, there are likely to be some side effect that are not discovered during the development and testing phases to enter the product environment.

3. Multi-browser

The biggest advantage of web apps is that they cross-platform across browsers, and users can access and use Web apps in different operating systems using different browsers. But this advantage also poses a big problem: a lot of browser compatibility testing is required. The larger the number of browsers being tested (now mainstream browsers include IE, Chrome, Firefox, Safari, etc., and there are many versions of each browser), the number and time of testing will multiply. This pain also makes it very difficult for many large Web applications to complete compatibility testing of most major browsers in a limited time period. If you do, you also need to pay a huge cost, such as adding more testers.

4. Responsive design (responsive Web design) testing

The popularity of mobile devices has led to a large number of users using mobile phones or tablets to use Web applications. Because mobile devices have a wide variety of resolutions, designers are also beginning to consider designing application interfaces for different resolutions, and responsive design (responsive Web design) is also born. But responsive design is difficult to test, basically only by hand, but also need to prepare a variety of resolution of the device or a variety of resolutions of the browser. The more resolution you need to test, the more time you have to test, and the higher the cost.

Here are two real web pages, many of which are different, and readers can try to find out how many. The answer is in the attached 2 figure.

Figure 2 Page 1

Figure 3 Page 2

What is visual perception testing-perceptual Testing

For the interface layout, the traditional test is the manual comparison of the design drawings and the product interface. When the interface has been modified, the human eye to check the changes (including correct and wrong changes), so that time-consuming and test results are not stable, because people are emotional. But we think that if an interface is validated and published for the first time, it is the right standard interface and an asset that contains the value of the manual test. In the next test, this part of the value should be retained and reused to reduce the time of a new test, thus enabling quick regression testing of the interface.

In order to solve the various problems mentioned above, visual perception testing was born. It uses the traditional method of binary comparison of images, combined with the idea of agile iterative development, resulting in an automated testing approach to the layout of the interface.

1. The visual perception test consists of several major test steps:

For product versions (product-line environment or product-class environment)

2. First to manually complete the first software version of the test and deployment on-line, the second version needs to be tested when the first version of all the interface.

For a new release (e.g. staging environment)

Then all the interfaces for the second version that needs to be tested are also performed.

3. Pairing URL (ignore hostname)

By pairing URLs, all are grouped by the same URL. Of course, sometimes the new interface will appear, sometimes the old interface will be deleted. For the new interface, a manual first verification test is required.

4. Pixel-level graphics comparison

For pixel-level comparisons after grouping and producing a diff chart. Sometimes for noise reduction, you can compare only the components that are part of the concern.

5. Manually view all the different

Finally, the test is completed by manually reviewing the differential chart report.

Some examples of visual perception testing

1.CSS of Change

For developers, CSS changes after the side effect is the most headache, the following shows the CSS after the revision of the page after the change.

Figure 4 CSS Changes

2. Changes in content

For testers, small changes to a large number of complex pages are hard to find, and the following shows how to use visual comparisons to find differences.

Figure 5 Changes in content 1

Figure 6 Changes in content 2

3. Change in event handling

For testers, some interfaces require a mouse click or hover to reveal them. The test for such an interface must be done manually. The following shows the difference between two versions of a selection box after a mouse click.

Figure 7 Change in event handling

4. Responsive design

For developers and testers, it would be a time-consuming and costly task to test a responsive design with devices with different resolutions, or to tweak the browser to a variety of resolutions. The following shows if the visual comparison is to check the responsive design.

Figure 8 Responsive design

Visual contrast-The last kilometer of UI Automation End-to-end testing

Visual perception testing in continuous delivery

For the traditional continuous delivery process:

Graph without visual perception of continuous delivery

The main difference in the continuous delivery process for the inclusion of visual perception testing is that the visual perception test is done in parallel with other automated tests before deployment.

Figure 10 includes continuous delivery of visual perception testing

The time for the interface regression test to implement the visual perception test

Figure 11 Interface Regression test time

Three visual perception testing tools

1.Mogotest

• From Mogotest, based on "cloud"
• Restful API Dev Test
• Multi-browser support
• Localization not supported
• Commercial

Mogotest is a commercially available product that provides a "cloud" test platform that allows users to access the tested pages and compare them on their platforms using a variety of different browsers. The main purpose is to test the compatibility between different browsers, not to test dynamic pages and so on.

2.DPXDT

• From Google, based on Python
• Only Phantomjs (Headless) is supported by default
• Restful API Dev Test
• Support Manual Authentication Report system
• Open Source, free

DPXDT is a Web service system developed based on Python and Phantomjs, where PHANTOMJS can be understood as a browser with no interface. Users can easily compare two pages with the RESTful API they provide, and it provides a very powerful reporting system. For all static pages of the web system is very suitable, but for manual navigation, such as the need for input, click or mouse hover and other operations to detect the interface, it is not supported by default, you need to modify it itself. But it also provides a way to deploy it conveniently to the GWS.

3.Viff

• From ThoughtWorks, based on Javascript/node
• Multi-browser support (Selenium webdriver)
• JAVASCRIPT/DSL Development Test
• Support Manual Authentication Report system
• Open Source, free
• Embedded test support: Smart TV, mobile phone, etc.

Viff is a local tool developed based on Nodejs and selenium. Write JavaScript code to invoke the selenium API and compare it in a real browser. So it is more suitable for dynamic Web systems because code can be written to simulate user input and click Operations. Because it uses selenium as the driver, he supports a variety of browsers, such as Ie,chrome,firefox. Because the latest selenium has added support for Android and iOS, Viff is also able to support browser testing on Android and iOS.

If it is difficult for you to build a multi-browser environment, such as the need to test ie8,ie9,ie10 at the same time, you can choose Browserstack. Browserstack is a commercial product that provides a multi-browser environment through the Web interface and API interface to customers for Web testing, VIFF can use its API for multi-browser. For Viff, because writing JavaScript code also requires a certain threshold, so for users without code ability to test static Web pages should choose Dpxdt, but if you have a certain ability to code, And hope to be in the current functional testing with visual perception testing or want to test the local interface, it is recommended to choose Viff. Now Viff is developing Web service functionality so that it can be deployed and used as a service in the future.

There are other visual perception testing tools, and here's a different one. On Viff's official website There is a comparison chart of several tools, interested readers can refer to: http://twers.github.io/Viff-Service/

Viff Demo

1. Installation

For Viff installation procedures, refer to the documentation Https://github.com/winsonwq/viff on their project.

To help you understand and learn viff, we have also developed a series of examples and demos, see Https://github.com/winsonwq/viff-examples, all of the following code from this project.

2. Install the demo site that needs to be tested

Download the code on the Https://github.com/winsonwq/viff-examples, simulate the product version of the site in Viff-examples/example/prod, simulate the site to be tested in Viff-examples /example/build inside. Because the demo site is static code, it can be deployed with any HTTP sever, for example, I use Nginx to deploy it locally on port 8000. After successful deployment, access to two test demo sites is available through Http://localhost:8000/example/build and Http://localhost:8000/example/prod.

3. Test with Viff's main API

For a completely new project, write the test code directly using Viff's main API, as follows:

The test results report is as follows:

1. First Test the home page, because there are no changes, so the test results are green, indicating that the product environment and test environment has not changed.

Figure 12 Presentation Results Report 1

2. Then enter commander.js in the Search input box and the result is that show 1 repository on the product version becomes showing 1 repositories on the beta version, so the test result is red. The report can then be changed immediately by the discovery and then manually audited for correctness of its modifications. This is obviously a bug in the beta version, and then it can be reported as a problem.

Figure 13 Presentation Results Report 2

3. Clear the Input box, then click Co in the Home page, then test the result or red. Careful review only found that generator in the beta version was changed to generators. It is very difficult to find a change in the letter S in so many content, but through Viff's test report, it is quick and easy to find the change.

Figure 14 Presentation Results Report 3

4. Test with Viff's client API

For a project that already has a functional testing, you do not need to re-develop the Viff test code, just call its client API in the functional test code, and you can also complete the visual perception test. Because the current VIFF only developed the JavaScript version of the client API, the following example uses JavaScript-based functional testing. Later Viff will initially provide client APIs based on other languages, such as Java,python and Ruby.

Code description, where two lines of green code indicate that the product environment and the test environment are functional testing, where all the blue code is functional testing code, where all the red part of the code is Viff's client API)

The test results report is the same as the 2nd step, as shown in Figure 12, figure 13 and Figure 14

5. Test the mobile web with Viff

Because of the selenium used by Viff, and selenium itself supports mobile web support, Viff is inherently included in this feature. For example, to test the iphone now, just change the 2nd step of the demo code to Firefox to iphone. (If you have questions about the test environment for selenium testing your iphone, see Selenium official documentation and selenium Iphonedriver:https://code.google.com/p/selenium/wiki/iphonedriver)

Several main usage scenarios for visual perception testing

Regression testing of 1.WEB application interface

2. Intelligent embedded Application Interface regression testing-such as smart TV applications, smart phone applications

3. Regression testing of Big data presentation

Summarize

Visual perception testing is an automated test method that appears in recent years to solve a lot of heavy manual interface regression tests. Not only does it help testers perform interface regression testing, but it also helps developers quickly perform side effect checks when refactoring or modifying common UI code, greatly reducing the time to test, and making regression testing of a large number of interfaces a reality, ultimately increasing the quality of the software, In particular, the greatest possible guarantee is the user experience of the software. It is hoped that in the near future, more and more complex interface systems will use visual perception testing to quickly complete their full regression testing to achieve true high quality, rapid and continuous delivery.

Reprint: http://www.uml.org.cn/Test/2014062410.asp

Visual perception Testing