Paper notes: Playing for Data:ground Truth from computer games

Playing for Data:ground Truth from computer games

ECCV 2016

　　Project Page: http://download.visinf.tu-darmstadt.de/data/from_games/index.html

　　 arXiv Paper: http://arxiv.org/pdf/1608.02192.pdf

　　

　　Abstract: This article is interesting, oh, from the video frame in the game to get new training data, why use this data? As we all know, recent computer vision relies heavily on massive amounts of data, but the data is always limited, so how do you get these fine-labeled data? Semi-supervised? Or is it done in a unsupervised way?? Oh, right, the last two days just played GTA5 game screen good Oh, yes, can you use those images as a new source of data? After all, now the game technology, has made the screen "Inverse Day". So this is the pixel-level segmented image map that is extracted from modern computer games. Although the game's source code and middle game operations are inaccessible, this article shows that the connection between image patches can be rebuilt between game and image hardware. This makes it possible to quickly transfer semantic labels between images in the game without having to access source code or content. Experimental results show that using these images to assist semantic segmentation can greatly improve the performance of segmentation: the model obtained by the game and the 1/3 Camvid training set exceed the precision obtained by using the Camvid training data.

The game is shown below, you can see the superb game screen, has been able to match the reality:

　　

　　The article motive: This article from the deep study needs the massive has the label data and the limited annotation ability contradiction as the starting point, starts to unfold to narrate this article. It takes a long time to fine-tune an image, where the author gives an example, and then compares the method presented in this article to a greater extent by reducing the label time.

This paper explores the use of computer games to build large pixel-level annotations to train semantic segmentation systems. The modern open game world, such as: GTA-5, watchdog, Killer 47 and so on, has restored a real world in height. Their idealization lies not only in the high quality material appearance and light variation simulation; they also relate to the content of the game world: external placement of objects and environments, real texture structures, vehicle movements, small objects that add details, interactions between players and the environment, and so on.

The size, appearance and behavior of the game world are much better than sandbox games because of its lack of these full elements. However, due to the fact that the operation and the contents of the game are almost invisible, the semantic annoation of detail is very challenging, and the problem can be solved by a technique called "detouring". We have added a wrapper between the game and the operating system to allow us to record, modify, and recreate the render commands. More crucially, it allows for the label to be passed over time and objects, sharing different resources.

In the same degree of labeling, our approach saves 3 orders of magnitude of time.

　　breaking the gap in data labeling :

　　

　　

　　

Paper notes: Playing for Data:ground Truth from computer games