Selenium Webdriver Mouse and keyboard event analysis and expansion (RPM)

This article summarizes the use of mouse and keyboard events in Selenium Webdriver, as well as the use of key combinations, and describes the extensions of keyboard events (keys not enumerated in the keys enumeration) that are not implemented in Webdriver. An example of an extended ALT+PRTSC key combination to intercept the currently active window and save the Clipboard image to a file.

Concept

When you use Selenium Webdriver for automated testing, you often simulate some of the mouse and keyboard behavior. such as the use of mouse click, double-click, right-click, drag-and-drop actions, or keyboard input, shortcut keys use, key combinations, such as analog keyboard operation. In Webderiver, there is a specialized class that is responsible for implementing these test scenarios, which is the Actions class that uses the Keys enumeration along with classes such as Mouse, Keyboard, Compositeaction, and so on in the process of using the class.

Second, in the actual test process, you may encounter some keys do not use the Actions, keys and other classes to achieve the situation. For example, by using the ALT+PRTSC key combination to capture the image of the currently active window of the screen, in the keys enumeration, because there is no PrtSc key, there is no way to simulate the action by the Action's KeyDown (keys).

Again in the automated test, you may encounter some attachments, file upload scenarios, or multiple file uploads, which can be implemented directly after Selenium2.0 using the SendKeys () method of the Webelement class.

The specific use of these cases is described below.

Mouse click action

There are several types of mouse click events:

Listing 1. Left mouse button click

actions = new Actions (driver), Action.click ();//left mouse button in the current position to do a click Operation Action.click (Driver.findelement (By.name ( Element))//left mouse button click on the specified elements

Listing 2. right mouse button click

Actions action = new actions (driver);  Action.contextclick ()//right mouse button in the current position to do a click Operation Action.contextclick (Driver.findelement (By.name (Element))// Right mouse click on the specified element

Listing 3. Mouse Double-click action

Actions action = new actions (driver);  Action.doubleclick ()//mouse in the current position to do double-click Action.doubleclick (Driver.findelement (By.name (Element))//mouse double-click the specified element

Listing 4. Mouse Drag action

Actions action = new actions (driver); Drag the mouse to drag and drop the source element to the location of the target element. Action.draganddrop (source,target);//Mouse drag action, drag and drop the source element to (Xoffset, yoffset) position, where Xoffset is the horizontal axis, yoffset for the ordinate. Action.draganddrop (Source,xoffset,yoffset);

In this drag process, has used the mouse combo action, first, the mouse click and hold (click-and-hold) The source element, and then perform the Mouse movement action (mouse move), move to the target element position or (Xoffset, Yoffset) position, and then perform the Mouse release action (mouse release). So the above method can also be broken into the following several execution actions to complete:

Action.clickandhold (source). Movetoelement (target). Perform ();  Action.release ();

Listing 5. Mouse Hover Action

Actions action = new actions (driver);  Action.clickandhold ();//mouse over the current position, click and not release Action.clickandhold (onelement);//hover over the location of the onelement element

Action.clickandhold (Onelement) This method is actually performed two actions, first, the mouse moved to the element onelement, and then clickandhold, so this method can also be written Action.movetoelement (onelement). Clickandhold ().

Listing 6. Mouse Move Action

Actions action = new actions (driver);  Action.movetoelement (toelement);//move the mouse over the toelement element to the point//move the mouse over the position of the element toelement (Xoffset, Yoffset),//Here (Xoffset, yoffs ET) is the (x, y) axis that starts with the upper-left corner of the element toelement (0,0). Action.movetoelement (Toelement,xoffset,yoffset)//At the current position of the mouse or (0,0) as the center to move to (Xoffset, yoffset) axis Action.movebyoffset (Xoffset,yoffset);

Action.movebyoffset (Xoffset,yoffset) It is important to note that if the xoffset is negative, the horizontal axis moves to the left, and the yoffset is negative to indicate that the ordinate is moving upward. And if these two values are larger than the current screen size, the mouse can only move to the edge of the screen and throw an movetargetoutofboundsexecption exception.

Mouse movement in the test environment is more commonly used in the scene is to obtain an element of the flyover/tips, the actual application of a lot of flyover only when the mouse moved to this element only appears, so this time by performing movetoelement (toelement) operation , you can achieve the desired results. However, according to my personal experience, this method does not work for some specific product icons, images and other flyover/tips, although moving the mouse to these icons can appear flyover when manually working, but when using webdriver to simulate this move operation, Although the method was executed successfully, but the flyover did not come out. So in the actual application, also need to the specific product page to do the corresponding processing.

Listing 7. Mouse Release Exercise

Actions action = new actions (driver);  Action.release ();//Release mouse

Keyboard emulation operation

For the simulation of the keyboard, the Actions class provides keyUp (Thekey), KeyDown (Thekey), SendKeys (Keystosend), and other methods to implement. The keyboard is operated with a normal keyboard and a modifier keyboard (Modifier keys, the following section will talk about the concept of modifier keys) two kinds:

1. For the general keyboard, use SendKeys (keystosend) can be implemented, such as key TAB, Backspace, and so on.

Listing 8. General Keyboard Simulation SendKeys (Keystosend)

Actions action = new actions (driver);  Action.sendkeys (Keys.tab);//Simulate press and release TAB key Action.sendkeys (keys.space);//Simulate press and release SPACEBAR/*** key action to issue a keyboard for an element, or an input operation, For example, you can use this method to enter a character in the input box. This method can also be split into: Action.click (Element). SendKeys (Keystosend). */Action.sendkeys (element,keystosend);

Note that in addition to the actions class has the SendKeys (Keystosend) method, the Webelement class also has a SendKeys (Keystosend) method, which is essentially the same for general input operations, with the difference being in the following points:

• The Call of SendKeys (Keystosend) in the Actions for the modifier key (Modifier keys) is not freed, that is, when Actions.sendkeys (Keys.alt) is called; Actions.sendkeys (Keys.control); Action.sendkeys (Keys.shift); , it is equivalent to calling Actions.keydown (Keystosend), and if you want to simulate pressing and releasing these modifier keys in a real-world application, you should call Action.sendkeys (keys. NULL) to complete this action.

• The second is that when the Actions of SendKeys (keystosend) are executed, the focus is not on the current element. So we can use SendKeys (keys.tab) to switch the focus of the element, so as to achieve the role of the selection element, the most common scenario is in the user name and password input process.
• 3rd, in Webdriver, we can use the Webelement class of SendKeys (Keystosend) to upload attachments, such as Element.sendkeys ("c:\\test\\uploadfile\\test.jpg ”); This action uploads test.jpg to the server, but uses:

Actions action = New actions (driver);  Action.sendkeys (element, "c:\\test\\upload\\test.jpg"); Action.click (Element). SendKeys ("c:\\test\\upload\\test.jpg");

This is not a successful upload, although Webdriver does not make an error while executing this statement, but does not actually upload the file. So to upload a file, you should still use the previous method.

2. For modifier keys (Modifier keys), they are generally used in combination with common key combinations. such as CTRL + A, ALT+F4, shift+ctrl+f and so on.

• Here first explain the concept of modifier keys, modifier keys are a keyboard or a special set of keys, when it is used in conjunction with the general key, used to temporarily change the general keyboard normal behavior. For individual presses the modifier key itself generally does not trigger any keyboard events. On a keyboard on a personal computer, there are several modifier keys: Shift, Ctrl, Alt (Option), AltGr, Windows logo, Command, FN (Function). However, in Webdriver, the general modifier key refers to the first three. You can click on the Wiki link below to learn more about modifier keys, Modifier key.
• Back to the above topic, the use of modifier keys in webdriver requires a KeyDown (Thekey), KeyUp (Thekey) method to operate.

Listing 9. Modifier key Method KeyDown (Thekey), KeyUp (Thekey)

Actions action = new actions (driver);  Action.keydown (Keys.control);//press the Ctrl key Action.keydown (keys.shift);//press the SHIFT key Action.keydown (Key.alt);//press the ALT key to act Ion.keyup (Keys.control);//Release The Ctrl key Action.keyup (keys.shift);//Release SHIFT key Action.keyup (Keys.alt);//Release ALT key

So to close the current active window through ALT+F4, you can do this by using the following statement: Action.keydown (Keys.alt). KeyDown (KEYS.F4). KeyUp (Keys.alt). Perform ();

And if it's for the letter key a,b,c,d like the keyboard ... Can be implemented using the following statement: Action.keydown (Keys.control). Sednkeys ("a"). Perform ();

In the Webdriver API, the parameters of the KeyDown (keys Thekey), KeyUp (keys Thekey) methods can only be modifier keys: keys.shift, Keys.alt, Keys.control, no will throw IllegalArgumentException exception. Next to the Action.keydown (Thekey) method call, if there is no display call Action.keyup (Thekey) or Action.sendkeys (keys.null) to release, this button will remain held down.

Use the Robot class to manipulate keys that are not enumerated.

1. In Webdriver, the keys list most of the non-alphanumeric keys on the keyboard, from F1 to F10,numpad0 to NUMPAD9, Alt\tab\ctrl\shift and so on, you can see all the keys enumerated by the keys by the following links, enum Keys. However, all keys on the keyboard are not listed, such as the letter keys A, B, C, D ... z, some symbolic keys such as: ' {}\[] ', ' \ ', '. ’、‘ ? ', ': ', ' + ', '-', ' = ',, ', ', and some less commonly used function keys such as PrtSc, Scrlk/nmlk. For the letter keys and symbol keys, we have mentioned earlier that we can use SendKeys ("a") and SendKeys ("/") to trigger these keyboard events. For some function combination keys, such as Fn + NMLK to close or open the number keys, or ALT+PRTSC to grab the current screen of the active window and save to the picture, through the Webdriver keys is not able to operate. This time we need to use the Java Robot class to implement this kind of key combination operation.

2. Let's take a look at ALT+PRTSC as an example of how Robot operates on the keyboard. such as code Listing 10.

Listing 10. To emit a combination of key actions by Robot

 /** * * @Description: This method is used to simulate sending a combination of ALT + PrtSc, and when the combo keyboard event executes, the active window on the screen is intercepted and stored on the Clipboard. The next step is to convert to an image object by reading the Clipboard data and saving it locally. * @param filename: The name of the image to save */public static void Sendcomposekeys (String filename) throws Exception {//Build Robot object, with  To operate the keyboard Robot Robot = new Robot ();  Simulates pressing the keyboard action by using the KeyEvent class to obtain the corresponding keyboard (ALT) virtual key code robot.keypress (Java.awt.event.KeyEvent.VK_ALT);  Press the PrtSC key robot.keypress (Java.awt.event.KeyEvent.VK_PRINTSCREEN); Release the keyboard action, when this action is completed, the simulation combination of ALT + PrtSC process has been completed,//At this time the screen Activity window is intercepted and into the Clipboard robot.keyrelease (java.awt.event.KeyEvent.VK_  ALT);  Get the system Clipboard instance Clipboard SYSC = Toolkit.getdefaulttoolkit (). Getsystemclipboard ();  Through the getcontents () method, the Clipboard contents can be obtained and deposited into the transferable object transferable data = Sysc.getcontents (null); if (data = null) {/*** Determines whether the object content obtained from the Clipboard is a Java image class, if it is converted directly to an image object. To this end, we are done with the process of issuing the key combination to the Crawl Activity window, then reading the Clipboard and storing the image object, and then we need to save the image object locally. */if (data.isdataflavorsupported (Dataflavor.imageflavor)) {Image image = (image) data. GettransferData (Dataflavor.imageflavor);  Writeimagetofile (image, FileName); }  }  }

Robot class to the keyboard processing is through the keyPress (int keycode), keyrelease (int keycode) method to achieve, wherein they need parameters are keyboard keys corresponding to the virtual key code, the value of the virtual key can be obtained through the KeyEvent class. In the Java API, the virtual key is interpreted as follows: The virtual key is used to report which key on the keyboard is pressed instead of the characters generated by the keystroke combination one or more times (such as "A" is generated by SHIFT + "a"). For example, holding down the Shift key generates a key_pressed event that KeyCode is Vk_shift, while pressing the ' a ' key will generate a Vk_a event keycode to key_pressed. When the ' A ' key is released, the key_released event keycode to Vk_a is fired. In addition, a key_typed event with a KeyChar value of ' a ' is generated. Pressing and releasing keys on the keyboard causes the following key events to be generated (in turn):

Key_pressed

Key_typed (generated only when valid Unicode characters can be generated.) ）

Key_released

So when you need to press the keyboard ALT+PRTSC key in the test, just execute the two keyPress () and a Keyrelease () method in Listing 10.

3. When the two keys are executed, the active window above the screen is saved to the Clipboard. If you need to save the local picture, you only need to read it from the Clipboard and write it locally via the JPEGImageEncoder class or the ImageIO class.

Listing 11. Use JPEGImageEncoder to save an Image object to a local

/** * * @Description: This method is used to save the image object to the local, mainly through the JPEGImageEncoder class to achieve the * * * Save * @param image : Image object to save * @param filename: The file name of the saved picture */public static void Writeimagetofile (image image, String filename) {TR Y {//Gets the width and height of the Image object, where the argument is null to not need to notify any observer  int width = image.getwidth (null);  int height = Image.  GetHeight (NULL);  BufferedImage bi = new BufferedImage (width, height, bufferedimage.type_int_rgb);  Draws an image by BufferedImage and saves it in its object Bi.getgraphics (). DrawImage (image, 0, 0, NULL);  Build image name and save path String name = const.directory + FileName + const.format;  File dir = new file (const.directory);  if (!dir.exists ()) {Dir.mkdir ();  } FileOutputStream out = new FileOutputStream (name);  @SuppressWarnings ("restriction") JPEGImageEncoder encoder = Jpegcodec.createjpegencoder (out);  Encoder.encode (BI);  Out.flush ();  Out.close ();  } catch (Exception e) {e.printstacktrace (); }  }

Listing 11 writes an image object to the local file stream through the JPEGImageEncoder class, noting that the image object is obtained in the following statement in code listing 10:

Clipboard SYSC = Toolkit.getdefaulttoolkit (). Getsystemclipboard ();  Transferable data = sysc.getcontents (null);  if (data = null) {  if (data.isdataflavorsupported (Dataflavor.imageflavor)) {  image image = (image) data  . Gettransferdata (dataflavor.imageflavor);  Writeimagetofile (image, FileName);  }  }

Listing 12. Use ImageIO to save an Image object to a local

/**  *  * @Description: Save the Image object as a local picture by using the ImageIO class * @param Image: The image object that needs to be saved * @param filename: file name */
   public static void SaveImage (image image, String fileName) throws Exception {  //Gets the height and width of the Image object int width = imag E.getwidth (null);  int height = image.getheight (null);  BufferedImage bi = new BufferedImage (width, height,  bufferedimage.type_int_rgb);  Graphics g = Bi.getgraphics ();      Draws an image by BufferedImage and saves it in its object g.drawimage (image, 0, 0, width, height, null);  G.dispose ();  File F = new file (fileName);  Writes an image to a file Imageio.write (bi, "JPG", f) via ImageIO;  }

Bulk uploading of files using SendKeys (keystosend)

Before Selenium2.0, uploading a file is a troublesome event, because clicking the Upload file control pops up the Windows window to provide the user's choice of files, but the window is already a component outside the browser, so the Selenium itself cannot be controlled, and You must use the Java Robot class to simulate the keyboard to operate the Clipboard to implement the upload function, and its instability. After Selenium 2.0, Webdriver solved the problem. As already mentioned before, directly using the Webelement class of SendKeys (Keystosend) method can be implemented file upload. However, if you want to upload files in bulk, using the Element.sendkeys ("C:\\test\\upload\\test1.txt", "c:\\test\\upload\\test2.txt" ...) method is also not possible, it can be executed, But actually no upload succeeded. At this time can be a circular way to achieve the bulk upload of files, listing 13 is my Baidu cloud upload files on the bulk of the test.

Listing 13. Uploading files in bulk

/**  *  * @Description: In the Baidu Cloud test file bulk upload function, mainly through the way to do a single * upload action, the landing process has been removed */  @Test public  void Test_ Mutiluploadfile () throws Exception {  System.out.println ("Upload start");  Gets the upload control element webelement Uploadbutton = driver.findelement (By.name ("Html5uploader"));  Build the upload file path, add the files that need to be uploaded to the charsequence array charsequence[] files = new charsequence[5];  Files[0] = "C:\\test\\test1.txt";  FILES[1] = "C:\\test\\test2.txt";  FILES[2] = "C:\\test\\test3.txt";  FILES[3] = "C:\\test\\test4.txt";  FILES[4] = "C:\\test\\test5.txt";  Loop lists each file path that needs to be uploaded, doing a single upload action for (charsequence file:files) {  uploadbutton.sendkeys (file);  }  Thread.Sleep (+);  System.out.println ("Upload End");  }

When execution finishes, the effect is 1.

Figure 1. Bulk Upload file Closing

In Selenium Webdriver, with the actions class and the Keys enumeration on the keyboard and mouse operation has been done very in place, combined with Java itself Robot, KeyEvent and other classes of use, basically can meet the work encountered in the application of the mouse keyboard operation.

Selenium Webdriver Mouse and keyboard event analysis and expansion (RPM)