DRL Frontier: Hierarchical deep reinforcement learning

1 Preface

If you have some knowledge of DQN, then we will know, DeepMind test of more than 40 games, there are several games no matter how training, the results are 0 of the game, that is, DQN completely invalid game, what game?

This game, for example, is called Montezuma ' s Revenge. This game is similar to Super Mary, where is it difficult? Advanced policies are required. To get the key, and then to open the door. This is obtained through prior knowledge for us. But it's hard to imagine how the computer perceives the content only through images. If you don't perceive it, the game will never be solved.

So this article:
Hierarchical deep reinforcement learning:integrating temporal abstraction and intrinsic motivation

Date: April 20, 2016
Source: arxiv.org

Try to solve this problem.

2 article Ideas

Its idea is simple, is to get a two-level neural network, the top layer for decision-making, to determine the next goal, the underlying for specific behavior.

I have to say, the idea is obvious (I've thought about it) but the point is

How to determine the intrinsic goal???

The author said such a passage in paper:
"We assume have access to a object detector that provides plausible object candidates."
To be blunt is to give a target artificially. And then we'll train. (essentially nesting of two dqn)

That's no point.

This can be trained from the intuitive sense.

But the meaning is relatively small.

Summary

This article is an exaggeration of the proposed level of DRL to solve the problem of sparse feedback, but in fact is not really a solution, the middle of the target is too artificial, not universal. It is also unlikely to develop a game object detection algorithm specifically for these games, as the author says.

However, in the case of the value of this article at the same time, it also has a certain significance. For example, for autonomous vehicles, before Nvidia fully end-to-end training to achieve automatic driving, but if the middle of adding an object detection as the top decision-making link, perhaps can greatly improve the level of control.

The same is true for Image caption. Object detection First, then enter RNN output text description.

However, individuals do not like this approach. Although it will work, but not intelligent.

DRL Frontier: Hierarchical deep reinforcement learning