Graphical scripting language Sikuli

 

Introduction                                                                                                                                                                                                                                                                                                

 

Sikuli is a novel graphic scripting language, or an alternative automated testing technology. It is very different from the commonly used automated testing technology (tools.

 

When you see the Sikuli script, you will be surprised. This is all possible ~! Script Addition ~~~

 

OK. After we are surprised, let's take a general look at this technology.

What is Sikuli?

Sikuli script automation, anything you see on the screen. It uses image recognition, recognition, and control GUI components. This is useful when there is a GUI internal orSource codeIs not easy.

Sikuli (in the Indian language of vijo in Mexico, it refers to the "eye of God") is a new programming technology developed by the Massachusetts Institute of Technology (MIT), allowing programmers to replaceCodeTo simplify the coding process. From its research perspective, it is a programming technology, but it can also be used for large-scaleProgramFor testing, the scripting program uses the Python language.

WHO is developing the Sikuli script?

Sikuli is an open source code initial user interface design organization for research projects at the Massachusetts Institute of Technology. It is now maintained and further coordinated with open sourceCommunityThe Sikuli lab was developed at the University of Boulder, Colorado. This is supported by part of the tparty project, part of the IIS-0447800 of the National Science Foundation Award. Issued under the Sikuli mit license.

 

Okay ~! You think Google's translation is too bad (but it is enough for me to know the meaning of this e-file cainiao), so you can go to the official website to see the English introduction:

Http://www.sikuli.org/

 

Sikuli can use python as a script language. Of course, if you are familiar with python, I believe you can get started with Sikuli very quickly. Of course, you can also use the Java API of Sikuli to run it in the Java environment.

 

Download and install

Are you interested in experiencing it with me?

Download the page to see the http://www.sikuli.org/download.html

Okay ~! This product has changed a lot. Last year, it was still a "simple" little software. Now it seems that the requirements have increased.

Sikuli tells us that there is currently only a 32-bit version, but this version should also run in 32-bit and 64-bit Windows systems. However, there are prerequisites:

 

1. Make sure that you have installed Java 6 JRE 32-bit (if it is Java 7 or 64-bit JRE, It is not supported)

2. Check whether you have uninstalled the previous Sikuli version (especially version 0.10.x)

3. Make sure that the system is restarted after you install JRE or uninstall the Sikuli of the old version.

 

Installation Steps:

1. On the download page provided above, click "Sikuli x-1.0rc3(r905316-win32.exe" to download and install it. The program is very small and can be ignored directly compared to tools such as qtp.

 

My installation path: I: \ Program Files (x86) \ Sikuli x

2. After the installation is complete, do not run it first. Because there is still a problem with it, we need to find the "Sikuli x r930" connection on the download page and download the ZIP file.

 

3. decompress the downloaded file sikuli-r930-win32.zip and get a SIKULI-IDE directory, replace all the files under the SIKULI-IDE directory with all the files under Sikuli X. (This is a bit of a problem !) This step aims to fix the latest bug in the r930 associated file, so replace the r905 (problematic version) File Association. I hope to officially go to the download version to fix it. You can see this articleArticleYou don't have to worry about downloading and installing it again.

 

 

Interface Introduction

The installation is complete. Run the following command to see what it looks like.

(Note that if the title bar of the window is Sikuli-r930, we use a replacement version)

 

The menu bar and toolbar are very simple. The following describes how to use these buttons.

 

Screen(Take screenshot): click this button to go to the screen state. Drag the guides to select the interface elements to be captured, and release the left mouse button. At the same time, it is automatically inserted to the current position of the cursor in the editing area. You can also use the shortcut key Ctrl + Shift + 2 (command + Shift + 2) to activate the status to complete the real-time display of controls such as pop-up menus and drop-down boxes. You can also use the main menu File> preferences to customize the shortcut key.

Insert Image(Insert image): You can also click this button to import existing PNG image files.

Create a screen area(Create region): click this button to enter the screen area selection status, drag and drop to locate the cross line to select the screen area. Release the left mouse button to insert the screen coordinate information of the selected area to the editing area.

Run(Run): Click to execute the current script. The shortcut key is Ctrl + R (command + r ).

Slow running(Run in slow motion): After clicking the button, execute the current script at a slow speed and explicitly mark each image search and positioning action with a red circle box to facilitate focus tracking during program debugging. The shortcut key is CTRL + ALT + R (command + ALT + r ).

 

Some common functions are listed in the left-side Navigation Pane. You can click a function name to quickly insert it to the editing area. If the function needs to be a parameter, it is automatically transferred to the screen status. The status bar below can be used to view the current row number and the level (column number) of tab indentation at the beginning of the row ).

The upper part of the area on the right is the script editing area. Our self-testing test script is compiled in this area.

The lower part is the prompt information. No matter whether the operation succeeds or fails, the corresponding prompt information is displayed.

 

 

Small Example

Generally, I understand the Sikuli interface. Here is a small example of automation. (The first official small example is to enter Hello world in the "Start" operation.) Here, we open the 360 speed browser (Baidu homepage by default), and enter Sikuli in the search box, click "Baidu" to search.

 Step 1,Click the desktop 360 speed browser icon.

Method 1: double-click the DoubleClick () function, and enter the desktop icon using the tool.

Method 2: Click double-click function DoubleClick () on the left side of the window to directly enter the status. Then, add the function to the script editing area.

 Step 2, We still use the second shortcut. Click the type (icon, text) function on the left side. The object captured this time is the input box of the Baidu homepage.

Because the focus is the center of the image by default, and the input position is on the left of the input box, you need to adjust the focus. Image in the text box.

 

 

Of course, in this window, we can also adjust the matching degree of the image. The lower the matching degree, the higher the fault tolerance capability. Here is a simple explanation:

We all know that although IE7 and IE8 are similar, they are still different. To make the script more portable. For example, the local computer uses the script recorded in IE7 and only IE 8 is transplanted to another computer. Because of the low precision, the script also runs IE8 as IE 7. Of course, in some cases, the downgrading is not good. For example, if there are two icons on the desktop, the precision adjustment is very low, and the script cannot identify who is who.

 

Step 3,This step is the same as the first step, but the function to be used here is to click () and click "Baidu.

The complete script is as follows:

 

Run this script! A sense of accomplishment is born. This is the motivation for your interest in this technology ..

 

 

Advantages and disadvantages 

Understanding the advantages and disadvantages of this tool will help you use this tool more correctly.

Advantages

1. Sikuli is small, convenient, and easy to use.

2. The Sikuli script can automatically search for anything you can see on the screen without going through the API compiler.

3. Suitable for various programs

4. There are no problems with standard and non-standard controls

5. The relative position concept is small and the requirement is not high (but the size of the object is affected)

6. One-time Encoding and multiple running (based on Java) 7. Support for Jython and Java, which is easier to expand by programming

 

Disadvantages 

1. The effect of the Resolution, color, size, and uniqueness of the image on the Program (if there are two identical ones, the specific one cannot be distinguished)

2. The Sikuli itself is not complete (in the development and upgrade stages) and there are many program bugs. It is still unknown whether it can continue to develop.

3. only recognize the icon of the current activity (only available on the current desktop)

4. the use case organization is provided, but there are bugs at present, and no research attempts are made for convenience.

5. Script storage occupies a large space.

6. Adjust the similarity one by one manually, resulting in heavy workload.

7. Differences between l0000200901t and l0000200902t (it is not easy to distinguish between them. It may need to be adjusted to similarity = 1.0, but the portability will be poor)

8. Currently, it is not suitable to be designed as a testing framework (useful example Organization function, but there are bugs and it has not been used yet)

9. The generation of the test report is weak if the message is used directly. It may be more powerful if Jython is used for programming (not yet researched)

 

 

--------------------

Your encouragement will allow me to further learn Sikuli and share my learning experience with you!