MonkenRunner's methods and suggestions for locating controls through HierarchyViewer (Appium/UIAutomator/Robotium companion), uiautomator

MonkenRunner's methods and suggestions for locating controls through HierarchyViewer (Appium/UIAutomator/Robotium companion), uiautomator
1. Background

When using MonkeyRunner, we often use the HierarchyViewer module under Chimchat to obtain information about the target control to help us test, but on MonkeyRunner's official website does not see the corresponding API description, above only the following three classes of API reference information (http://developer.android.com/tools/help/MonkeyDevice.html)

• MonkeyDevice
• MonkeyImage
• MonkeyRunner

So here, I will try to sort out the API usage provided by HierarchyViewer and make corresponding suggestions based on the practice. First of all, please refer to all the available public methods provided by this class. The content is not much:

It can be seen that the methods provided in the HierarchyViewer class are mainly used to locate controls, including obtaining controls based on IDs and obtaining control positions on the Screen Based on controls. However, there are other methods that we will describe along with. After all, there is not much content. We are still keeping up with several articles in this article, taking the NotePad that comes with the SDK as the experimental goal, to see how to locate the Menu Entry "Add note" in Menu Options under NotesList. The following figure shows the target device interface obtained through the HierarchyViewer tool:
2. findViewById (String id) 2.1 example

targetDevice = MonkeyRunner.waitForConnection()'''     public ViewNode findViewById(String id)      * @param id id for the view.     * @return view with the specified ID, or {@code null} if no view found.'''viewer = targetDevice.getHierarchyViewer()button = viewer.findViewById('id/title')text = viewer.getText(button)print text.encode('utf-8')

2.2 Analysis and Suggestion the purpose of this API is to obtain a ViewNode object representing the user control through the control ID. This is the first example.

• Once the MonkeyRunner connects to the device, it will immediately obtain a MonkeyDevice object representing the target test device. We use this device object to control the device's
• Note that the format of the id to be filled in is different from that obtained by UIAutomatorViewer. For details, see the "android:" string added before ResourceId in UIAutomatorViewer:
• The ViewNode object returned by this method represents the target control and has a large number of control-related attributes. Due to space issues, we will not detail them here. We will write another article to describe its usage in the future. In this article, we know that it represents the target control.
• The reason for converting to UTF-8 encoding when printing at last has something to do with Jython's default encoding format, the specific description and Workaround please view: http://www.haogongju.net/art/1636997

3. findViewById (String id, ViewNode rootNode) Example 3.1

'''     public ViewNode findViewById(String id, ViewNode rootNode)      * Find a view by ID, starting from the given root node     * @param id ID of the view you're looking for     * @param rootNode the ViewNode at which to begin the traversal     * @return view with the specified ID, or {@code null} if no view found.     '''iconMenuView = viewer.findViewById('id/icon_menu')button = viewer.findViewById('id/title',iconMenuView)print "Button Text:",text.encode('utf-8')

3.2 This method is an overload of the above method. In addition to specifying the ID, you also need to specify a rootNode, which refers to the parent control of the known control, there is no limit on the parent hierarchy. Why do we need this method? We can imagine this situation: two controls on the same interface have the same ID, but one of them has a parent control at a certain level and starts to split. Then, we can specify rootNode as one of the parent controls in the parent control (not included) to the target control (not included) path to precisely locate the target control we need. For example, our example clearly states that what we need is the "id/title" control under the parent control "id/icon_menu" (see hierarchyviewer in the background.
4 getAbsolutePositionOfView (ViewNode node) 4.1 example

'''    public static Point getAbsoluteCenterOfView(ViewNode node)     * Gets the absolute x/y center of the specified view node.     *     * @param node view node to find position of.     * @return absolute x/y center of the specified view node.     */'''point = viewer.getAbsoluteCenterOfView(button)print "Button Absolute Center Position:",point

4.2 Analysis and Suggestion the purpose of this API is to locate the absolute coordinates on the screen in the upper left corner of a known ViewNode control. I have practiced the controls in our normal APP, but it is no problem. However, there is a situation where special attention should be paid to: This is invalid for the controls under Menu Options! The output of the last code in the preceding example is (3, 18). In fact, you don't need to know that this is not the absolute coordinate value relative to the coordinate (0, 0) in the upper left corner of the screen. It offsets this pixel, you really can be my lab machine, HTC Incredible S. How can this data be obtained? Actually, according to my understanding (it's really just my own understanding. If it's not correct, please describe the details for my reference ), the definition of this function should be "Get the DecorView from the top layer (for details about DectorView, please refer to an article I previously posted," Android DecorView analysis ") the offset coordinate from the coordinate in the upper left corner to the target control, but the coordinates of the DecorView at the top are generally starting from (0, 0. For example, I think the top FrameLayout represents DecorView, or the entire form.
If my opinion is correct, this will be well resolved. Please refer to the absolute coordinates of FrameLayout at the top of Menu Option (0,683)
The absolute coordinate of Add note is (3,701)

The subtraction of the two is absolutely consistent with our output results.
5. getAbsoluteCenterOfView (ViewNode node) Example 5.1

'''    public static Point getAbsoluteCenterOfView(ViewNode node)     * Gets the absolute x/y center of the specified view node.     *     * @param node view node to find position of.     * @return absolute x/y center of the specified view node.     */'''point = viewer.getAbsoluteCenterOfView(button)print "Button Absolute Center Position:",point

5.2 Analysis and Suggestion the purpose of this method is to obtain the absolute coordinate value of the center point of the target ViewNode control, but it is not applicable to the controls under Menu Options. For details, see section 3rd.
The following two methods are not used to locate the control and are recorded for reference. 6. getFocusedWindowName () 6.1 Example

'''    public String getFocusedWindowName()     * Gets the window that currently receives the focus.     *     * @return name of the window that currently receives the focus.'''window = viewer.getFocusedWindowName()print "Window Name:",window.encode('utf-8')

6.2 resolution

In fact, it is to obtain the packageName/activityName of the window currently opened. The output is consistent with the information detected by the HierarchyViewer tool, so we guess it is used in the same way.

Output:

HierarchyViewer monitoring information:

7. visible (ViewNode node) 7.1 example

'''    public boolean visible(ViewNode node)      * Gets the visibility of a given element.     * @param selector selector for the view.     * @return True if the element is visible.'''isVisible = viewer.visible(button)print "is visible:",isVisible

Check whether the control is visible. There is no better resolution.

8. Test code

from com.android.monkeyrunner import MonkeyRunner,MonkeyDevicefrom com.android.monkeyrunner.easy import EasyMonkeyDevice,Byfrom com.android.chimpchat.hierarchyviewer import HierarchyViewerfrom com.android.hierarchyviewerlib.models import ViewNode, Windowfrom java.awt import Point#from com.android.hierarchyviewerlib.device import #Connect to the target targetDevicetargetDevice = MonkeyRunner.waitForConnection()easy_device = EasyMonkeyDevice(targetDevice)  #touch a button by id would need thistargetDevice.startActivity(component="com.example.android.notepad/com.example.android.notepad.NotesList")#time.sleep(2000)#invoke the menu optionsMonkeyRunner.sleep(6)targetDevice.press('KEYCODE_MENU', MonkeyDevice.DOWN_AND_UP);'''     public ViewNode findViewById(String id)      * @param id id for the view.     * @return view with the specified ID, or {@code null} if no view found.'''viewer = targetDevice.getHierarchyViewer()button = viewer.findViewById('id/title')text = viewer.getText(button)print text.encode('utf-8')'''     public ViewNode findViewById(String id, ViewNode rootNode)      * Find a view by ID, starting from the given root node     * @param id ID of the view you're looking for     * @param rootNode the ViewNode at which to begin the traversal     * @return view with the specified ID, or {@code null} if no view found.     '''iconMenuView = viewer.findViewById('id/icon_menu')button = viewer.findViewById('id/title',iconMenuView)print "Button Text:",text.encode('utf-8')'''    public String getFocusedWindowName()     * Gets the window that currently receives the focus.     *     * @return name of the window that currently receives the focus.'''window = viewer.getFocusedWindowName()print "Window Name:",window.encode('utf-8')'''    public static Point getAbsoluteCenterOfView(ViewNode node)     * Gets the absolute x/y center of the specified view node.     *     * @param node view node to find position of.     * @return absolute x/y center of the specified view node.     */'''point = viewer.getAbsoluteCenterOfView(button)print "Button Absolute Center Position:",point'''    public static Point getAbsolutePositionOfView(ViewNode node)     * Gets the absolute x/y position of the view node.     *     * @param node view node to find position of.     * @return point specifying the x/y position of the node.'''point = viewer.getAbsolutePositionOfView(button)print "Button Absolute Position:", point'''    public boolean visible(ViewNode node)      * Gets the visibility of a given element.     * @param selector selector for the view.     * @return True if the element is visible.'''isVisible = viewer.visible(button)print "is visible:",isVisible

9. Add the source code of the HierarchyViewer class for reference.

/* * Copyright (C) 2011 The Android Open Source Project * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * *      http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package com.android.chimpchat.hierarchyviewer;import com.android.ddmlib.IDevice;import com.android.ddmlib.Log;import com.android.hierarchyviewerlib.device.DeviceBridge;import com.android.hierarchyviewerlib.device.ViewServerDevice;import com.android.hierarchyviewerlib.models.ViewNode;import com.android.hierarchyviewerlib.models.Window;import org.eclipse.swt.graphics.Point;/** * Class for querying the view hierarchy of the device. */public class HierarchyViewer {    public static final String TAG = "hierarchyviewer";    private IDevice mDevice;    /**     * Constructs the hierarchy viewer for the specified device.     *     * @param device The Android device to connect to.     */    public HierarchyViewer(IDevice device) {        this.mDevice = device;        setupViewServer();    }    private void setupViewServer() {        DeviceBridge.setupDeviceForward(mDevice);        if (!DeviceBridge.isViewServerRunning(mDevice)) {            if (!DeviceBridge.startViewServer(mDevice)) {                // TODO: Get rid of this delay.                try {                    Thread.sleep(2000);                } catch (InterruptedException e) {                }                if (!DeviceBridge.startViewServer(mDevice)) {                    Log.e(TAG, "Unable to debug device " + mDevice);                    throw new RuntimeException("Could not connect to the view server");                }                return;            }        }        DeviceBridge.loadViewServerInfo(mDevice);    }    /**     * Find a view by id.     *     * @param id id for the view.     * @return view with the specified ID, or {@code null} if no view found.     */    public ViewNode findViewById(String id) {        ViewNode rootNode = DeviceBridge.loadWindowData(                new Window(new ViewServerDevice(mDevice), "", 0xffffffff));        if (rootNode == null) {            throw new RuntimeException("Could not dump view");        }        return findViewById(id, rootNode);    }    /**     * Find a view by ID, starting from the given root node     * @param id ID of the view you're looking for     * @param rootNode the ViewNode at which to begin the traversal     * @return view with the specified ID, or {@code null} if no view found.     */    public ViewNode findViewById(String id, ViewNode rootNode) {        if (rootNode.id.equals(id)) {            return rootNode;        }        for (ViewNode child : rootNode.children) {            ViewNode found = findViewById(id,child);            if (found != null) {                return found;            }        }        return null;    }    /**     * Gets the window that currently receives the focus.     *     * @return name of the window that currently receives the focus.     */    public String getFocusedWindowName() {        int id = DeviceBridge.getFocusedWindow(mDevice);        Window[] windows = DeviceBridge.loadWindows(new ViewServerDevice(mDevice), mDevice);        for (Window w : windows) {            if (w.getHashCode() == id)                return w.getTitle();        }        return null;    }    /**     * Gets the absolute x/y position of the view node.     *     * @param node view node to find position of.     * @return point specifying the x/y position of the node.     */    public static Point getAbsolutePositionOfView(ViewNode node) {        int x = node.left;        int y = node.top;        ViewNode p = node.parent;        while (p != null) {            x += p.left - p.scrollX;            y += p.top - p.scrollY;            p = p.parent;        }        return new Point(x, y);    }    /**     * Gets the absolute x/y center of the specified view node.     *     * @param node view node to find position of.     * @return absolute x/y center of the specified view node.     */    public static Point getAbsoluteCenterOfView(ViewNode node) {        Point point = getAbsolutePositionOfView(node);        return new Point(                point.x + (node.width / 2), point.y + (node.height / 2));    }    /**     * Gets the visibility of a given element.     *     * @param selector selector for the view.     * @return True if the element is visible.     */    public boolean visible(ViewNode node) {        boolean ret = (node != null)                && node.namedProperties.containsKey("getVisibility()")                && "VISIBLE".equalsIgnoreCase(                        node.namedProperties.get("getVisibility()").value);        return ret;    }    /**     * Gets the text of a given element.     *     * @param selector selector for the view.     * @return the text of the given element.     */    public String getText(ViewNode node) {        if (node == null) {            throw new RuntimeException("Node not found");        }        ViewNode.Property textProperty = node.namedProperties.get("text:mText");        if (textProperty == null) {            // give it another chance, ICS ViewServer returns mText            textProperty = node.namedProperties.get("mText");            if (textProperty == null) {                throw new RuntimeException("No text property on node");            }        }        return textProperty.value;    }}

10. For more information, see the following practices for locating controls in different frameworks:

• Android Control Positioning practices and suggestions for Robotium (Appium/UIAutomator companion)
  
• UIAutomator practices and suggestions for locating Android controls (Appium companion article)
  
• Practices and Suggestions on positioning methods of Appium based on various FindElement controls of Android