The correct way for AI to Jump to WeChat: detailed explanation of the Auto-jump algorithm, aiauto-Jump

Correct posture for AI to perform a hop: detailed explanation of the Auto-Jump Algorithm for a hop, aiauto-jump
Author: Xiao Taihong anjie
Peking University | Graduate School of Mathematics
Authorized quantum bit Editing and Publishing
For reprint, contact the original author.

Recently, a small game has jumped around the country. From a child to a big child, it seems that everyone is jumping.

As an AI programmer at the NO (zhi) Institute (hui) Not (ban) can (zhuan), we are wondering if we can use artificial intelligence (AI) and computer vision (CV) to play this game?

As a result, we have developed the Auto-Jump algorithm to redefine the correct posture for a hop. Our algorithm is far beyond the human level, the speed and accuracy far surpass all the known algorithms. It can be said that it is the state-of-the-art in the hop field. Next we will introduce our algorithms in detail.

The first step of the algorithm is to obtain the mobile phone screen and control the touch operation of the mobile phone. Our github repository details the configuration methods for Android and IOS Mobile Phones.

GitHub address:
Https://github.com/Prinsphield/Wechat_AutoJump

You only need to connect your phone to your computer and complete the configuration according to the tutorial. After obtaining the screen, it is a simple visual problem. We need to find the location of the villain and the center of the table to be jumped next time.

Green points represent the current location of the villain, and red points represent the target location.

Multi-scale Search)

This problem can be solved in many ways. In order to make it rough and fast, I used a multi-scale search method. I randomly found a picture and pulled the villain out, just like below.

In addition, I noticed that the size of the screen is slightly different for the villain, So I designed a multi-scale search, matched with different sizes, and finally selected confidence score) the highest.

The code for multi-scale search is long --

Let's give it a try. The effect is good. It should be said that it is fast and good. I have never made any mistakes in finding a villain in all my experiments.

However, the center at the bottom of the position box is not the location of the villain. The actual location is above that.

Similarly, you can use this method to search for the target table, but we need to collect some different tables, such as round, square, convenience store, manhole cover, and prism. Due to the large quantity and multi-scale reasons, the speed will slow down. At this time, we need to find a solutionAcceleration.

First, you can notice that the target position is always on the top of the villain's position. So one thing you can do is to discard all the parts below the villain's position after finding the villain's position, which can reduce the search space. But this is not enough. We need to further explore the stories in the game.

Small people and target countertops are basically about the symmetrical location of the center of the screen. This provides a good idea to narrow down the search space. Assume that the screen resolution is (1280,720) and the bottom position of the villain is (h1, w1). The location of the center symmetric point is (1280-h1, 720-w1 ), in a square with a side length of 300 centered on this point, we can search for the target location in multiple scales, and then we can quickly get it right.

The result is as follows. The blue box is the search area (300,300), the red box is the found table, and the center of the rectangle is the coordinates of the target point.

Fast-Search)

Be careful when playing games. We can find that if the villain jumps to the center of the table, there will be a white dot in the center of the next table, as shown in the previous figure.

People who are more careful will find that the RGB value of white points is (245,245,245), which allows me to find a very simple and efficient way, that is, to directly search for this white point, note that the white point is a connected area, and the number of pixels with a pixel value of (245,245,245) is stable between and, so we can use this to directly locate the target location.

This method can only be used when the previous jump to the center, but it does not matter. We can try this method every time, without having to consider multi-scale search.

At this point, our method can run very well, basically a permanent motive. The following figure shows how to use my mobile phone for about an hour and a half. I jumped 859 times. Our method correctly calculates the location of the villain and the target location, but I chose the dog belt, because the phone card is no longer working.

Here is an example video. You are welcome to watch it!

Is it over now? What is the difference between us and amateurs? The following shows the actual academic time. non-combat personnel should be quickly evacuated!

CNN Coarse-to-Fine Model

Considering the limitations of the screen capture solution (the image pixels are damaged due to compression of the WebDriverAgent, which is no longer the original pixel value, the reason is unknown, you are welcome to give suggestions for improvement ~) Fast-search is not available. To be compatible with multi-resolution devices, we use a convolutional neural network to build a faster and more robust target detection model. The following describes data collection and preprocessing, And the coarse model, fine model. cascade introduces our algorithms.

Data collection and preprocessing

Based on our extremely accurate multiscale-search and fast-search models, we collected 7 experimental data, with a total of about 3000 screens, each with a target location annotation, for each graph, we conduct two different preprocessing methods, which are used to train the coarse model and the fine model respectively. The following describes two different preprocessing methods.

Coarse model data preprocessing

Each image has only the central position of the screen, that is, the position of the person and the target object. Therefore, the upper and lower parts of each image are meaningless, therefore, we cut off the collected 1280720 images along the x direction and cut them up and down. Only the 320720*640 images in the center are retained as the training data.

In the game, each time a villain falls into the center of the target object, a white dot appears in the center of the next target object.

Considering that fast-search generates a large amount of data with white spots in the training data, in order to prevent the interference of white dots on network training, we perform the de-white operations on each graph, fill the White Point area with solid pixels around the white point.

Fine model data preprocessing

To further improve the accuracy of the model, we have set up a dataset for the fine model. For each graph in the training set, we can take a 320*320 size piece near the target point as the training data,

To prevent the network from learning trivial results, we add a random offset of 50 pixels to each graph. Fine model data is also de-white.

Coarse Model

We regard this issue as a regression problem. The coarse model uses a convolutional neural network to return to the target location,

After ten hours of training, the coarse model achieved a 6-pixel precision in the test set. The actual test accuracy is about 10 pixels. On the testing machine (MacBook Pro Retina, 15-inch, Mid 2015, 2.2 GHz Intel Core i7) the inference time is 0.4 seconds.

This model can easily get scores of more than 1 K, which is far beyond the human level and the vast majority of automatic algorithms. Daily entertainment is sufficient. However, you think that's a big mistake ~

Fine Model

The fine model structure is similar to the coarse model. The fine model has a relatively large number of parameters and serves as a refine operation for the coarse model.

After ten hours of training, the precision of the fine model test set reaches 0.5 pixels. The actual test accuracy is about 1 pixel, and the inference time on the test machine is 0.2 seconds.

Cascade

The overall precision is about 1 pixel, and the time is 0.6 seconds.

Summary

To address this problem, we use AI and CV technology to propose a complete solution suitable for IOS and Android devices. users with a slight technical background can achieve successful configuration and operation, we propose three algorithms to solve this problem: Multiscale-Search, Fast-Search, and CNN Coarse-to-Fine. These algorithms work together to achieve Fast and accurate Search and jump, you can adjust the Skip parameter slightly for your device to achieve "permanent motivation ".

Speaking of this, it seems that we can announce that our work is terminate, and the game jumps over!

Friendly reminder: Moderate game benefits and addiction to games. The pleasure of technical means lies in that technology itself is not ranked in the game rankings. I hope you can treat the game rankings rationally and the technologies proposed in this article, play your own life with games

Disclaimer: the algorithm and open source code proposed in this article comply with the MIT open source protocol. All consequences arising from the use of this algorithm for commercial purposes shall be borne by the user himself.

Contributors

Xiao Taihong

Xiaotaihong@126.com

An Jie

Jie.an@pku.edu.cn

Click"Read original"To Unlock more articles by the author.

You can also directly participate in the discussion ~

For more information, see the original author.

-End-

Join community

The 13th Group of the quantum AI community began to recruit. We welcome students interested in AI to join the group with the assistant qbitbot5;

In addition (Self-driving, CV, NLP, machine learning) Recruiting engineers and researchers in related fields.

Add the assistant number qbitbot5 to the group and note the keywords of the group ~ After the application is approved, we will invite you to the group. (Strict review by professional groups, please forgive me)

Sincere recruitment

Quantum bit is recruiting Editors/reporters working in Zhongguancun, Beijing. We look forward to talented and enthusiastic people joining us! For details, please reply to "recruitment" on the QbitAI dialog page.

Quantum bitQbitAI toutiao.com contracted

Tracking AI technologies and new product trends