Android TV Tile App Automation framework two modifications (based on Uiautomator)

Brief introduction

The project has always been manual testing, plus has been a TV class application, many automation tools are not for the TV class project to do a good fit, so only to do it yourself. Mainly for the particularity of the project has been partially modified, not necessarily applicable to other projects. (no JSON file or software name is provided for privacy)

Pain Point

1. The difficulty of non-standard controls

The different tiles obtained through Uiautomatorviewer, the properties are all the same (except the coordinate points), means that it is not available through the ID and Class+index way, the Text property is empty, there is no way to get uiobject through Bytext way, Highly customizable tiles, making automation difficult, if through coordinates point, too pit Daddy, can't cross equipment, or pit.

2.TV class Application no touch Operation
TV class Android program, mainly for the remote control, which is to receive the keyevent, so the touch event is not so comprehensive, in order to closest to the user, or choose to use key to do automation.

Schema module

Parsing module

Parses the JSON file format from the server, wraps it into the entity class Cellinfo, and returns a list containing the magnet information. A magnet corresponds to a cellinfo, and a cellinfo needs to extract the information there

• X-coordinate
• Y-coordinate
• The tab category page that belongs to
• Description label for each magnet

So the corresponding definition is as follows:

Package Launcherclick.model; Public classCellinfo {PrivateString label;PrivateString tab; PublicStringGettab() {returntab } Public void Settab(String tab) { This. tab = tab; }Private intXPrivate intY PublicStringGetlabel() {returnLabel } Public void SetLabel(String label) { This. label = label; } Public int GetX() {returnX } Public void SetX(intx) { This. x = x; } Public int GetY() {returnY } Public void sety(intY) { This. y = y; }}

Parsing is using the third-party open Source Library Org.json, the corresponding parsing code is as follows:

Package Launcherclick;Import Java. Util. ArrayList;Import Java. Util. List;import org. JSON. Jsonarray;import org. JSON. Jsonobject;Import Utils. Ioutils;Import Utils. Println;Import Launcherclick. Model. Cellinfo;public class Ametroparse {public static list<cellinfo> startparse (string url) {String str = Ioutils. Readfromnet(URL);if (str==null) {return null;} jsonobject jsonobject = new Jsonobject (str);list<cellinfo> list = new arraylist<cellinfo> ();Jsonarray _tabs = Jsonobject. Getjsonarray("Tabs");for (int i =0; I<_tabs.length (); i++) {String tab = _tabs. Getjsonobject(i). getString("Label");Jsonarray _cells = _tabs. Getjsonobject(i). Getjsonarray("Cells");for (int j=0; J<_cells.length (); j + +) {Cellinfo cellinfo = new Cellinfo ();Cellinfo. SetX(_cells. Getjsonobject(j). Getjsonobject("Location"). GetInt("x"));Cellinfo. Sety(_cells. Getjsonobject(j). Getjsonobject("Location"). GetInt("Y"));Cellinfo. SetLabel(_cells. Getjsonobject(j). Getjsonobject("Content"). getString("Label"));Cellinfo. Settab(tab);List. Add(Cellinfo);}} return List;}}

Coordinate conversion module

This part of the work is to extract and process the parsed entity class from the previous step, the main processing is to calculate the magnet movement according to the x y coordinate, the maximal value of X and the classification page to do the migration of cross-classification movement and some other processing.
It is simple to calculate the offset by x y, just a simple calculation of the distance:

         for(Cellinfo cellinfo:List) {if(Cellinfo.getlabel (). Contains (label) && Cellinfo.gettab (). Contains (tab)) {intx = Cellinfo.getx ();inty = cellinfo.gety ();NewPrintln ("x:"+ x +"  "+"y:"+ y +"  "+"offset"+ _offset);//x direction                 for(inti = x + _offset; i >1; i--) {NewPrintln ("→"+"Count is"+ (x + _offset));                Keyright (); }//Y direction                 for(inti = y; i >1; i--) {NewPrintln ("↓");                KeyDown (); }            }        }

Well, there's an offset, the cross-classification, the offset is the maximum value of the magnet under each category, the X-value of the last magnet in each category is the maximum, and the x of all the magnets under the current classification, if it is greater than the next one, is placed in front, and the value below is used to compare the x value below. Forget what sort of algorithm this is ... I'm so embarrassed ... Of course, you can also use collections's own algorithm.

    privateinttogicOffset(String tab) {        int0;        List<CellInfo> list = AMetroParse                .startParse("我是隐藏的接口");        for (CellInfo cellInfo : list) {            if (cellInfo.getTab().contains(tab)) {                int x = cellInfo.getX();                lineLenth = lineLenth >= x ? lineLenth : x;            }        }        return lineLenth;    }

Then by the above sorting, you can get the offset of each classification, so for the subsequent classification of the tile, you can know the real position of the tile in the whole, so you can start to move.

         for(Cellinfo cellinfo:List) {if(Cellinfo.getlabel (). Contains (label) && Cellinfo.gettab (). Contains (tab)) {intx = Cellinfo.getx ();inty = cellinfo.gety ();NewPrintln ("x:"+ x +"  "+"y:"+ y +"  "+"offset"+ _offset);//x direction                 for(inti = x + _offset; i >1; i--) {NewPrintln ("→"+"Count is"+ (x + _offset));                Keyright (); }//Y direction                 for(inti = y; i >1; i--) {NewPrintln ("↓");                KeyDown (); }            }        }

Picture comparison Module

From the idea of monkeyrunner, the principle is to calculate the length and width of two bitmap, and then extract the pixel value of each pixel inside, if the same, similarity +1, and finally divided by the total pixel value (such as 1280x720), so that the similarity can be converted to a measurable value , then judge this interface is not when I need to click, only need to screen the current picture and the expected picture contrast, the similarity reached 100 when it is considered correct. Of course, it also provides features for local comparisons, such as exception hints that do not get data. The code below provides a variety of overloaded methods and two assert judgments.

 PackageUtils;ImportJunit.framework.Assert;ImportAndroid.graphics.Bitmap;ImportAndroid.graphics.BitmapFactory; Public  class imagecompare {     Public  Static void assertbitmapequal(Bitmap bitmap0, Bitmap bitmap1) {/** * Hundred picture same assertion-.- */Assert.assertequals ( -, Imagecompare (BITMAP0,BITMAP1)); } Public  Static void assertbitmapnotequal(Bitmap bitmap0, Bitmap bitmap1) {/** * 0% Picture Same assertion */Assert.assertequals (0, Imagecompare (BITMAP0,BITMAP1)); } Public Static int Imagecompare(String path0,string path1) {/** * Provides direct comparison by path * /Bitmap bitmap0 = Bitmapfactory.decodefile (Path0); Bitmap Bitmap1 = Bitmapfactory.decodefile (path1);returnImagecompare (BITMAP0,BITMAP1); } Public Static int Imagecomparechild(Bitmap bitmap0, Bitmap Bitmap1,intXintYintWidthintHeight) {/** * Crop the sub-graph and compare, mainly in order to solve the pull dynamic data is different, but the local hint unchanged comparison scene. */Bitmap bitmap00 = Bitmap0.createbitmap (bitmap0, x, y, width, height); Bitmap bitmap01 = Bitmap1.createbitmap (Bitmap1, x, y, width, height);returnImagecompare (BITMAP00,BITMAP01); } Public Static int Imagecompare(Bitmap bitmap0, Bitmap bitmap1) {/** * Comparison of the main function * can only compare the same length and width of the picture, unequal return-1 failure * similarity to 1~100 * principle is to extract each pixel comparison, the whole map similarity depends on the number of pixels, so still The more accurate * /        intpicpct =0;intPiccount =0;intPiccountall =0;NewPrintln ("begin to compare");if(Bitmap0 = =NULL|| Bitmap1 = =NULL) {NewPrintln ("null bitmap");return-1; }if(Bitmap0.getwidth ()! = Bitmap1.getwidth () | | bitmap0.getheight ()! = Bitmap1.getheight ()) {return-1; }NewPrintln ("width is:"+ bitmap1.getwidth () +"Height is:"+ bitmap1.getheight ()); for(intj =0; J < Bitmap1.getwidth (); J + +) { for(inti =0; I < bitmap0.getheight (); i++) {if(Bitmap0.getpixel (j, i) = = Bitmap1.getpixel (j, i))                {piccount++;            } piccountall++; }        }intresult = (int) (((float) piccount)/Piccountall * -);NewPrintln (Piccount +"/"+ Piccountall);NewPrintln ("Similarity is:"+ result);returnResult }}

Exceptions && Other modules

Defines some exception classes, the main function is used to prompt, the role of this hint will be used later.
Added a way to start the app and close the app, using the shell command.

Runtime.getRuntime().exec("am start -n 我是包名隐藏者");

Initialization method, mainly for the exception can be a key to start again, as well as the tile recovery function, there are some other small processing is not much to say, the following began to build continuously.

Continuous integration with automation

Or that sentence, the automation of non-continuous integration is not automated, so here is the automation integration based on Jenkins, in fact, uiautomator do integration is very easy, just put the jar package in a fixed directory, and then the shell command execution is done.

Then the scheduled task to choose their own, build a failed email alert here is set, above my custom exception class, when the console output my exception class, then will think not to pass, and then trigger email reminders.

So far, a more complete process has been completed, then in this Framework team members (just me) can be more into a two-level page to improve the automation use cases.

Android TV Tile App Automation framework two modifications (based on Uiautomator)