Automated Acceptance Test with Selenium

ArticleDirectory

• Automated Acceptance Test with Selenium

Automated Acceptance Test with Selenium

How to Use selenium test tools for Ruby on Rails and Ajax applicationsProgramTest Functions

Document options

Send this page as an email

Discussion

ExampleCode

Expand Tomcat applications

Download the latest version of IBM Open-Source J2EE application server was Ce V1.1

Level: Intermediate

Christian Hellsten (christian.hellsten@fi.ibm.com), IT expert, IBM

January 04, 2006

Acceptance Tests (also called functional tests) are used to test the functions of manual tasks. However, manual tests of these tasks may take a long time and are prone to human errors. In this article, the author shows architects, developers, and testers how to use selenium testing tools for automated acceptance testing. Automated Testing can save time and eliminate errors made by testers. An example is provided to demonstrate how to apply selenium to projects using Ruby on Rails and Ajax in reality.

Web application acceptance testing often involves some manual tasks, such as opening a browser and executing the operations described in a test case. However, manual tasks are prone to human errors and time-consuming. Therefore, it is a good practice to automate these tasks as much as possible to eliminate human factors. Therefore, selenium and other testing tools are available. Selenium helps you automate acceptance tests to build software that is more rigorous, reliable, and easy to maintain.

The acceptance test, also known as the Black Box test and functional test, is to test and test whether the application can follow the stakeholders (stakeholder) it is a method to run with functional requirements, non-functional requirements, and other important requirements. Acceptance testing is a supplement to unit testing and combination testing. The latter two are usually written using the xunit framework. The acceptance test can also be used.Programming LanguageBut selenium and other similar tools, such as fitnesse, also support writing tests in a tool-specific document format.

Acceptance testing differs from unit testing and combination testing in the following ways:

• An application is tested as a complete end-to-end entity, instead of testing a class or a group of classes, just like unit testing and combination testing.
• The acceptance test is executed on the user interface (such as a browser), rather than on the Web application interface.
• Developers who write test cases do not necessarily know the internal structure of the application, so they are also called Black Box tests. Non-technical users can also write acceptance tests.

Background

Before discussing selenium, I would like to introduce some background knowledge about the following three topics, because these topics are not the subject of this article, but are closely related to this article:

• Continuous Integration
• Ajax
• Ruby/Ruby on Rails

Continuous Integration

Continuous integration is designed to automate building and testing processes so that they can run automatically once or multiple times a day rather than manually once a month. The biggest benefit of using continuous integration is that code changes are automatically integrated on a regular basis. If the system is damaged and the build fails, continuous integration tools such as Apache continuum and luntbuild can automatically send an email to the Team (see references ).

Ajax

Ajax is the abbreviation of Asynchronous JavaScript and XML, a term newly created for a fairly old technology. The main idea behind Ajax is that because you only need to update some pages instead of the whole page, web applications can respond to user operations more quickly.

Ajax introduces more complexity into web applications, which is also reflected in testing. This is because Ajax uses JavaScript and asynchronous HTTP requests to update page content as its name indicates. Each browser is slightly different from other browsers in implementation. Selenium is a good tool for testing and detecting these differences because it can run in most popular browsers.

Ruby/Ruby on Rails

Ruby is an open-source interpreted scripting language for fast and easy object-oriented programming. It provides a large number of libraries, which are easy to use and scalable and portable. This language was created by yukihiro "Matz" Matsumoto to allow programmers to focus more on their tasks and get rid of grammar troubles.

Rails is a full-stack and open-source Ruby Web framework created by David Heinemeier Hansson. Rails aims to write less code for real-world applications and is easier than J2EE or XML. All layers work seamlessly together, so you can use one language to write everything from templates to control flows and even business logic. Rails uses yaml instead of xml configuration files and annotation reflection and runtime extensions. There is no compilation phase here-the Program will run directly after modification.

Back to Top

What is selenium?

Selenium is an acceptance test tool specially written by thoughtworks for Web applications. According to the selenium homepage, the biggest advantage of using selenium compared with other test tools is:

The selenium test runs directly in the browser, just as the real user does. Selenium testing can be run in Internet Explorer, Mozilla, and Firefox on Windows, Linux, and macdeskshand. Other test tools cannot cover so many platforms.

There are many other benefits to Using Selenium and running tests in a browser. The following are two major benefits:

• By writing a selenium test script that imitates user operations, you can test applications from the perspective of end users.
• By running tests in different browsers, it is easier to detect incompatibility between browsers.

The core of selenium, also knownBrowser BotIs written in JavaScript. This allows the test script to run in a supported browser. Browser bot is responsible for executing the commands received from the test script. The test script is either written in HTML table layout or in a supported programming language.

Selenium applies to the following browsers:

Internet Explorer
Mozilla
Firefox
Safari

Windows XP
6.0
1.6 +, 1.7 +
0.8 +, 0.9 +, 1.0

Red Hat Linux
1.6 +, 1.7 +
0.8 +, 0.9 +, 1.0 +

Mac OS X 10.3
Not Supported
1.6 +, 1.7 +
0.8 +, 0.9 +, 1.0 +
1.3 +

Back to Top

Selenium command

With the selenium command, the script writer can describe the operations performed by browser BOT in the browser. These commands can be divided into two types --Action)AndAssertion):

• The operation simulates the interaction between the user and the web application. For example, clicking a button and entering a form are common user operations. You can use the selenium command to automate these operations.
• Assertions verify the expected results of a command. Common assertions include verifying that the page content or current location is correct.

You can find a complete list of available commands on the selenium website (see references ).

Back to Top

Selenium Mode

Selenium can be used in two modes:Test runnerAndDriven. These two modes differ in complexity and writing methods. Writing driven test scripts is more complex because they are written in programming languages. However, if you use advanced dynamic programming languages such as Python or Ruby, there is little difference in complexity.

The biggest difference between the two modes is that if the Driven Script is used, some tests run outside the browser, and if the test runner script is used, the tests run completely in the browser.

Both test runner and driven test cases can be integrated with continuous integration tools.

Back to Top

Test runner Mode

Selenium test runner script, also knownTest Case)Is written in HTML language through a simple table layout, as shown in Listing 1.

Listing 1. Structure of selenium Test Cases

<Table border = "1"> <tr> <TD> First Command </TD> <TD> Target </TD> <TD> value </TD> </tr> <tr> <TD> second command </TD> <TD> Target </TD> <TD> value </TD> </tr> </table>
 

The test runner script is usually deployed on the same server as the tested application (AUT. This is because browser Bot uses JavaScript to simulate user operations. These scripts are run in a restricted sandbox environment. If you need to bypass these restrictions, you can use a proxy.

The test runner script uses the same test suite and test case concepts as the xunit framework. Test Cases and commands are executed in sequence as they appear in the test suite and test cases. In listing 1:

• The first column containsCommandOrAssertions.
• The second column contains commands or assertionsTarget). You can use one of the multiple supported component locators to specify the target. The component ID or name is usually used, but the XPath and Dom locators are also supported.
• The third column containsValue. For exampleTypeCommand, this column may be the expected value of a text field.

Even for non-technical personnel, the test runner script is easy to read and write. When you open the example in Listing 1 in a browser, you will get a table like this:

First Command
Target
Value

Second command
Target
Value

Next, I will describe how to use commands and assertions to write a simple but complete test case.

Test Case instance

When you run the test script in Listing 2, it performs the following operations:

1. Open the address change page by entering/change_address_form.html.
2. InAddress_fieldIn the text boxBetelgeuse State Prison.
3. ClickSubmit. Note: Use XPath to findSubmitButton, which causes the form data to be sent to the server.
4. Verify that the page contains textAddress change successful.

Listing 2. Examples of using commands and assertions in Test Cases

<Table> <tr> <TD> open </TD> <TD>/change_address_form.html </TD> </tr> <TD> type </TD> <TD> address_field </TD> <TD> Betelgeuse State Prison </TD> </tr> <TD> clickandwait </TD> <TD> // input [@ name = 'submit '] </TD> </tr> <TD> verifytextpresent </TD> <TD> address change successful </TD> </tr> </table>
 

Test Suite

To fully test the application, more than one test case is required. This is why selenium uses the test suite. Test suites are used to group test cases with similar features so that they can run in sequence.

Like test cases, the test suite is written in simple HTML tables. The default test suite name for selenium execution is testsuite.html. Listing 3 shows a test suite that tests applications like a common user. Note: The test suite uses a table containing only one column. Each row in the table points to a file containing a test case.

Listing 3. Test Suite example

<Table> <tr> <TD> test suite for the whole application </TD> </tr> <TD> <a href = "test_main_page.html"> access Main Page </a> </TD> </tr> <TD> <a href = "test_login.html"> login to application </a> </TD> </tr> <tr> <TD> <a href = "test_address_change.html"> change address </a> </TD> </tr> <TD> <a href =" test_logout.html "> logout from application </a> </TD> </tr> </table>
 

Next, I will focus on the driven test script.

Back to Top

Driven Mode

Driven selenium scripts are written in a variety of supported programming languages-currently available Java, Ruby, and Python drivers. These scripts are run in a separate process outside the browser. The task of the driver is to execute the test script and communicate with the browser bot running in the browser to drive the browser. The communication between the driver and the browser Bot uses selenese, a simple connection language specific to selenium.

Driven scripts are more powerful and flexible than test runner scripts and can be integrated with the xunit framework. The disadvantage of driven scripts (compared with the test runner script) is that such scripts are more complicated to write and deploy. This is because the driver must execute the following tasks:

• Start the server.
• Deploy the tested application (AUT ).
• Deploy the test script.
• Start the browser.
• Send the command to browser bot.
• Verify the result of the Command executed by browser bot.

The Driven Script is more dependent on the runtime environment of the application. For example, the Java driver uses an embedded jetty or Tomcat instance to deploy the tested application. At present, some people are already working to integrate selenium into Ruby on Rails, but at the time of writing this article, this integrated version has not yet been released.

Listing 4 is taken from a driven test script using the ruby driver. Note that I omitted the steps for starting the server and the browser. The test script code is almost as simple as the test runner script.

Listing 4. Example of using the ruby driver

 
.. Puts selenium. Open ('/logout.html') puts selenium. verify_location ('/index.html ')..
 

Back to Top

Real-world needs

In the next two sections (real-world requirements and real-world use cases), I will describe how to use selenium in real-world scenarios, and write selenium test cases for a simple stock quotation Viewer application written using Ruby on Rails and a little Ajax technology. Although this application is written in Ruby on Rails, this example can also be applied to any web application, because the test script is written in HTML in the test runner mode. This example application was tested using Ruby 1.8.3 and Ruby on Rails 0.14.2, but it may also be able to use older or newer versions.

If Linux is available, Ruby is usually included in the release. RunRuby-VCheck your version. For most platforms, you can find a ruby release on the http://www.ruby-lang.org.

The next step is to install Ruby on Rails through the rubygems packaging system. Therefore, you only need to executeGem install rails -- include-dependencies. Some additional steps must be performed on some platforms, so visit the Ruby on Rails website for more details.

At the time of writing this article, the current available selenium version is 0.6. I have integrated it into the sample application (see the download section), and my approach is to download the selenium core package from the http://selenium.thoughtworks.com, copy the folder named selenium to the folder used for static content. In the Ruby on Rails application, the folder name is public. In J2EE Web applications, you can place the selenium folder in the root directory or war archive file of the Web application.

The last step is to download the sample application. Obtain this package from the download section. Decompress the application and open a command prompt. And then transfer it to the directory where the application is extracted. To start the application, runRuby script/Server. You can see that rails is successfully started, as shown in figure 1.

Figure 1. Run Ruby on Rails from a command prompt

Back to Top

Real-world use cases

In this section, I will list the use cases of the sample application. With these simplified use cases, you can write acceptance tests that simulate the steps performed by the user, and verify that the results of these steps are consistent with expectations. The stock quote application implements the following four use cases:

• Login
• View stock
• View stock details
• Exit

The code for implementing these use cases has been compiled; you can find the code in the app directory, and the test case is in the public/selenium/tests folder.

Logon case

Most people know how the logon page works-enter the user name and password, and then submit the data to the server. If the credential is valid, you can log on successfully and see the protected resources. In the example application, this test case contains the following user operations and assertions that must be converted into a selenium test case:

1. Click the logon link.
2. Verify that the system requires the user to log on.
3. Enter the user name.
4. Enter the password.
5. Click Log On.
6. Verify that the logon is successful.

Figure 2 shows the selenium test cases for these requirements. Note that I take a screen snapshot after running the test. The green arrow indicates the successfully verified assertions.

Figure 2. log on to and view stock Test Cases

View stock Test Cases

View A company list on the stock page. The test cases for this page are very simple, so they are included after the previous test case. This test case verifies whether the current location is/main/list_stocks and whether the page contains textClick on a company name to View Details.

View stock details Use Cases

The use case for viewing stock details is triggered on The View stock page. When a user clicks the mouse over a company name, an Ajax request to the server is triggered. The server response includes the company's detailed information, which is inserted into the current page without the need to reload the complete page. The test cases for this use case perform the following user operations:

1. Click company nameAcme oil.
2. Verify that the company details are displayed on the page.
3. Click company nameAcme Automotive.
4. Verify that the company details are displayed on the page.

Because Ajax is used, requests occur asynchronously. In general web applications, everything is usually synchronized, so this poses a different challenge. You can test the Ajax function just like other functions. The only difference is that selenium must be paused, waiting for the Ajax command to complete. To do this, you can usePauseTo wait for the completion of the Ajax command. In addition, Joseph Moore mentioned in his recent blog post thatWaitforvalueAndWaitforconditionCommand to replace the pause command (see references ).

Figure 3 shows the requirements for converting to selenium use cases.

Figure 3. View stock details Test Cases

Note:PauseCommand: You must use these commands to wait for the asynchronous request to complete and update the page content. If the pause is not performed within 500 milliseconds, the test fails (as shown in Figure 4 ).

Figure 4. Failed to view stock details Test Cases

PauseThe command also tests the non-functional requirements of Ajax functions. 500 milliseconds is a good value for the pause command, because Ajax requests should be executed and completed quickly. You can try to remove it.PauseCommand to see how the result is. If the test fails on your machine, try to increase the value to 1000 milliseconds.

Exit use case

Exit the use case is easy to implement. Simply put, there are only two steps:

1. Click exit.
2. Verify whether the application is successfully exited.

Figure 5 shows the final test case.

Figure 5. Exit Case

All tests are added to the default test suite shown on the left of Figure 6.

Figure 6. test suite of the sample application

Execution test suite

The last thing to do is to execute the test suite in Mozilla Firefox and Microsoft Internet Explorer. To this end, openHTTP: /localhost: 3000/selenium/testrunner.htmlAnd then clickAllButton. Failed test cases and assertions will be marked in red, but here, all use cases in both browsers should be successfully completed (see figure 6 ). Note: I use Mozilla Firefox 1.0.7 and Internet Explorer 6.0.

You can also debug the test suite in one step, which means selenium will execute the test suite very slowly, so that each step of the test suite will be visible when it is executed in a browser.

Back to Top

Conclusion

Selenium is another useful and important tool in the toolbox of software engineers, designers and testers. By combining the tool with the continuous integration tool, the team can automate the acceptance test and build better software because they can discover bugs more easily, earlier, and more frequently. Another advantage of selenium is that it can save time, so that developers and testers do not have to spend time on manual tasks that can (or should) be automated, this allows the team to focus on more valuable activities.