"CV" icml2015_unsupervised learning of Video representations using LSTMS

Unsupervised learning of Video representations using LSTMS

Note Here:it ' A learning notes on the new LSTMS architecture used as an unsupervised learning the by of video representations.

(More unsupervised learning related topics, you can refer to:

Learning temporal embeddings for Complex Video analysis

Unsupervised learning of Visual representations using Videos

unsupervised Visual representation learning by Context prediction)

link:http://arxiv.org/abs/1502.04681

Motivation:

-Understanding temporal sequences is important for solving many video related problems. We should utilize temporal structure of videos as a supervisory signal for unsupervised learning.

Proposed model:

In this paper, the author proposed three models based on LSTM:

1) LSTM Autoencoder Model:

This model is composed of parts, the encoder and the decoder.

The encoder accepts sequences of frames as input, and the learned representation generated from encoder is copied to Deco Der as initial input. Then the decoder should reconstruct similar images as input frames in reverse order.

(This was called unconditional version, while a conditional version receives last generated output of decoder as input, sho WN as the dashed boxes below)

Intuition: The reconstruction work requires the "Network to capture information" about the appearance of objects and the background, th Is is exactly the information so we would like the representation to contain.

2) LSTM Future Predictor Model:

This model is similar with the one above. The main difference lies in the output. Output of this model was the prediction of frames that come just after the input sequences. It also varies with conditional/unconditional versions just like the description above.

Intuition: In order to predict the next few frames correctly, the model needs information about which objects is present and how the Y is moving so, the motion can be extrapolated.

3) A Composite Model:

This model combines ' input reconstruction ' and ' future prediction ' together to form a more powerful model. These modules share a same encoder, which encodes input sequences into a feature vector and copy them to different Dec Oders.

Intuition: This is only encoder learns representations that contain is only static appearance of OBJECTS&BACKGR Ound, but also the dynamic informations like moving objects and their moving pattern.

"CV" icml2015_unsupervised learning of Video representations using LSTMS