Awesome Recurrent neural Networks
A curated list of resources dedicated to recurrent neural networks (closely related to deep learning).
Maintainers-jiwon Kim, Myungsub Choi
We have pages for other topics:awesome-deep-vision, awesome-random-forest
Contributing
Please feel free-to-pull requests, email myungsub Choi ([e-Mail protected]) or join our chats to add links.
Sharing
- Share on Twitter
- Share on Facebook
- Share on Google Plus
- Share on LinkedIn
Table of Contents
- Codes
- Theory
- Lectures
- Books/thesis
- Network variants
- Surveys
- Applications
- Language Modeling
- Speech recognition
- Machine translation
- Image captioning
- Video captioning
- Question Answering
- Image Generation
- Turing Machines
- Robotics
- Datasets
Codes
- Theano-python
- Simple IPython tutorial on Theano
- Deep Learning Tutorials
- RNN for semantic parsing of speech
- LSTM Network for sentiment analysis
- keras:theano-based Deep Learning Library
- Theano-rnn by Graham Taylor
- Passage:library for text analysis with Rnns
- Caffe-c++ with Matlab/python wrappers
- Torch-lua
- Char-rnn by Andrej Karpathy:multi-layer Rnn/lstm/gru for training/sampling from Character-level language models
- LSTM by Wojciech Zaremba:long Short term Memory Units to train a language model on word level Penn Tree Bank DataSet
- Oxford by Nando de Freitas:oxford computer science-machine learning practicals
- ETC.
- Rnnlib by Alex graves:c++ based LSTM Library
- RNNLM by Tomas mikolov:c++ based simple code
- Neuraltalk by Andrej karpathy:numpy-based rnn/lstm implementation
- Gist by Andrej Karpathy:raw NumPy code, implements an efficient batched LSTM
Theorylectures
- Stanford NLP (cs224d) by Richard Socher
- Lecture Note 3:neural Network Basics
- Lecture Note 4:rnn language models, Bi-directional RNN, GRU, LSTM
- Oxford machine Learning by Nando de Freitas
- Lecture 12:recurrent Neural Networks and LSTMS
- Lecture: (Guest lecture) Alex Graves on hallucination with Rnns
Books/thesis
- Alex Graves (2008)
- Supervised Sequence labelling with recurrent neural Networks
- Tomas Mikolov (2012)
- Statistical Language Models based on neural Networks
- Ilya Sutskever (2013)
- Training Recurrent neural Networks
- Richard Socher (2014)
- Recursive deep Learning for Natural Language processing and computer Vision
Network variants
- Bi-directional RNN [Paper]
- Mike Schuster and Kuldip K. Paliwal, bidirectional recurrent neuralNetworks, Trans. On Signal Processing 1997
- LSTM [Paper]
- Sepp Hochreiter and Jurgen Schmidhuber, Long short-term Memory, neural Computation 1997
- Multi-dimensional RNN [Paper]
- Alex Graves, Santiago Fernandez, and Jurgen Schmidhuber, multi-dimensional recurrent neural Networks, ICANN 2007
- GRU (Gated recurrent Unit) [Paper]
- Kyunghyun Cho, Bart van Berrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi bougares, Holger Schwenk, and Yoshua Bengio, Learning Phrase Representations using RNN Encoder-decoder for statistical machine translation, arxiv:1406.1078/ EMNLP 2014
- GFRNN [PAPER-ARXIV] [PAPER-ICML] [supplementary]
- Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, Yoshua Bengio, Gated Feedback recurrent neural Networks, arxiv:15 02.02367/ICML 2015
Surveys
- Klaus Greff, Rupesh Kumar Srivastava, Jan Koutnik, Bas R steunebrink, Jurgen schmidhuber,lstm:a Search Space Odyssey, AR xiv:1503.04069
- Zachary C. Lipton, A Critical Review of recurrent neural Networks for Sequence learning, arxiv:1506.00019
- Andrej karpathy, Justin Johnson, Li Fei-fei, visualizing and understanding recurrent Networks, arxiv:1506.02078
- Rafal Jozefowicz, Wojciech Zaremba, Ilya Sutskever, an empirical exploration of recurrent Network architectures, ICML, 201 5.
Applicationslanguage Modeling
- Tomas Mikolov, Martin karafiat, Lukas burget, Jan "Honza" Cernocky, Sanjeev Khudanpur,recurrent neural Network based L Anguage Model, Interspeech [Paper]
- Tomas Mikolov, Stefan kombrink, Lukas burget, Jan "Honza" Cernocky, Sanjeev Khudanpur,Extensions of recurrent neural N Etwork Language Model, ICASSP [Paper]
- Stefan Kombrink, Tomas Mikolov, Martin karafiat, Lukas burget, recurrent neural Network based Language Modeling in Mee Ting recognition, Interspeech [Paper]
Speech recognition
- Geoffrey Hinton, Li Deng, Dong Yu, George E. Dahl, Abdel-rahman Mohamed, Navdeep jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N. Sainath, and Brian Kingsbury, deepneural Networks for acoustic Modeling in Speech recognition , IEEE signam processing Magazine [Paper]
- Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton, Speech recognitionwith deep recurrent neural Networks, a RXIV:1303.5778/ICASSP [Paper]
Machine translation
- Univ. Montreal [Paper]
- Kyunghyun Cho, Bart van Berrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi bougares, Holger Schwenk, and Yoshua Bengio, Learning Phrase Representations using RNN Encoder-decoder for statistical machine translation, arxiv:1406.1078/ EMNLP 2014
- Google [Paper]
- Ilya Sutskever, Oriol vinyals, and Quoc v. Le, Sequence to Sequence learning with neural Networks, arxiv:1409.321 5/nips 2014
- Univ. Montreal [Paper]
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio, neural machine translation by jointly learning to Align and transl Ate, ARXIV:1409.0473/ICLR 2015
Image captioning
- Baidu + UCLA [Web] [Paper-arxiv1], [PAPER-ICLR]
- Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, and Alan L. Yuille, Explain Images with multimodal recurrent neural networks< /c1>, arxiv:1410.1090
- Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan L. Yuille, deep captioning with multimodal recurrent Neural Networks (M-RNN), ARXIV:1412.6632/ICLR 2015
- Univ. Toronto [Paper] [Web Demo]
- Ryan Kiros, Ruslan Salakhutdinov, and Richard S. Zemel, Unifying visual-semantic embeddings with multimodal neural Lan guage Models, Arxiv:1411.2539/tacl 2015
- Berkeley [Web] [Paper]
- Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Dar Rell, long-term recurrent convolutional Networks for Visual recognition and Description, ARXIV:1411.4389/CVPR 2 015
- Google [Paper]
- Oriol vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan, Show and tell:a neural Image Caption Generator, ARXIV:1411.4555/CVPR 2015
- Microsoft [Paper]
- Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mi Tchell, John c. Platt, Lawrence Zitnick, and Geoffrey Zweig, from captions to Visual conceptsand back, arxiv:141 1.4952/CVPR 2015
- Microsoft [PAPER-ARXIV], [PAPER-CVPR]
- Xinlei Chen, and C. Lawrence Zitnick, learning a recurrent Visual representation for Image Caption Generation
- Xinlei Chen, and C. Lawrence Zitnick, Mind's eye:a recurrent Visual representation for Image Caption Generation, CVPR 2015
- Univ. Toronto + Univ. Montreal [Web] [Paper]
- Kelvin Xu, Jimmy Lei Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua be Ngio, Show, attend, and tell:neural Image Caption Generation with Visual Attention, ARXIV:1502.03044/ICML 2015
- Idiap + EPFL + Facebook [Paper]
- Remi Lebret, Pedro O Pinheiro, and Ronan Collobert, phrase-based Image captioning, ARXIV:1502.03671/ICML 2015
- Baidu + UCLA [Paper]
- Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan L. Yuille, learning like a child:fast novel Visual C Oncept learning from sentence descriptions of Images, arxiv:1504.06692
Video captioning
- Berkeley [Web] [Paper]
- Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Dar Rell, long-term recurrent convolutional Networks for Visual recognition and Description, ARXIV:1411.4389/CVPR 2 015
- UT Austin + UML + Berkeley [Paper]
- Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, and Kate Saenko, translating Videos To Natural Language Using deep Recurrent neural Networks, arxiv:1412.4729
- Microsoft [Paper]
- Yingwei Pan, Tao Mei, Ting Yao, Houqiang Li, and Yong Rui, Joint Modeling embedding and translation to Bridge Video an D Language, arxiv:1505.01861
- UT Austin + Berkeley + UML [Paper]
- Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue, Raymond Mooney, Trevor Darrell, and Kate Saenko, Sequence to Se Quence--video to Text, arxiv:1505.00487
Question Answering
- MSR + Virginia Tech. [Web] [Paper]
- Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi-Parikh, V Qa:visual Question answering, ARXIV:1505.00468/CVPR sunw:scene Understanding Workshop
- MPI + Berkeley [Web] [Paper]
- Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz, Ask Your neurons:a neural-based approach to answering Questions About Images, arxiv:1505.01121
- Univ. Toronto [Paper] [Dataset]
- Mengye Ren, Ryan Kiros, and Richard Zemel, Image Question answering:a Visual Semantic embedding Model and A New Datas Et, arxiv:1505.02074/icml deep Learning Workshop
- Baidu + UCLA [Paper] [Dataset]
- Hauyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, and Wei Xu, is you talking to a machine? Dataset and Methods for multilingual Image Question answering, arxiv:1505.05612
Image Generation
- Karol Gregor, Ivo Danihelka, Alex Graves, Danilo J. Rezende, and Daan Wierstra, draw:a recurrent neural Network for I Mage Generation, ICML [Paper]
- Angeliki Lazaridou, Dat T. Nguyen, R. Bernardi, and M. Baroni, unveiling the Dreams of Word embeddings:towards Langua Ge-driven Image Generation, arxiv:1506.03500
- Lucas Theis and Matthias bethge, generative Image Modeling Using Spatial Lstms,arxiv:1506.03478
Turing Machines
- A.graves, G. Wayne, and I. Danihelka., neural Turing machines, arXiv preprint arxiv:1410.5401 [Paper]
- Jason Weston, Sumit Chopra, Antoine Bordes, Memory Networks, arxiv:1410.3916. [Paper]
- Wojciech Zaremba, Ilya sutskever, reinforcement learning neural Turing machines,arxiv:1505.00521. [Paper]
Robotics
- Marvin Zhang, Sergey Levine, Zoe McCarthy, Chelsea Finn, Pieter abbeel, Policy Learning with continuous Memory states For partially observed robotic Control, arxiv:1507.01273. [Paper]
Datasets
- Image captioning
- Flickr 8k
- Flickr 30k
- Microsoft COCO
- Image Question Answering-all based on MS COCO images
- VQA
- Image QA
- [Multilingual Image QA]: In Chinese, with 中文版 translation
Awesome Recurrent neural Networks
