Summary of deep learning

Comments from Xinwei: This article is another overview of translation on the deeplearning website. It mainly describes some papers and algorithms that have been used in the toolbox.

 

Deep Learning is a new field in ml research. It is introduced into ml to bring ml closer to its original goal: AI. View A Brief Introduction to machine learning for AI and an introduction to deep learning algorithms.

 

Deep Learning is about learning multiple levels of representation and abstraction that help explain data, examples, sounds, and text. For more information about deep learning algorithms, see:

• The monograph or review paper Learning deep ubuntures for AI (Foundations & Trends in machine learning, 2009 ).
• The icml 2009 Workshop on learning Feature Hierarchies webpage has a list of references.
• The Lisa public Wiki has a reading list and a bibliography.
• Geoff Hinton has readings from last year's nips tutorial.

This article mainly introduces some of the most important deep learning algorithms and demonstrates how to run them with theano. Theano is a python library that makes it easier to write deep learning models. It also provides some options for training them on the GPU.

 

The summary of this algorithm has some prerequisites. You should first know about Python and be familiar with numpy. Since this summary is about how to use theano, you should first read theano basic tutorial. Once you finish this, read ourGetting startedChapter --- it will introduce concepts, definitions, datasets, and methods for optimizing models using random gradient descent.

 

Supervised Learning algorithms can be read in the following order:

1. Logistic Regression-Using theano for something simple
2. Multilayer perceptron-Introduction to Layers
3. Deep convolutional Network-A simplified version of lenet5

Unsupervised and semi-supervised learning algorithms can be read in any order (auto-encoders can be read independently of RBM/DBM ):

• Auto encoders, denoising autoencoders-Description of autoencoders
• Stacked denoising auto-encoders-Easy Steps into unsupervised pre-training for deep nets
• Restricted Boltzmann machines-Single Layer generative RBM Model
• Deep belief networks-Unsupervised generative pre-training of stacked RBMS followed by supervised fine-tuning

For the mcrbm model, we have a new review on sampling from the energy model:

• HMC sampling-Hybrid (aka Hamiltonian) Monte-Carlo sampling with scan ()

 

 

 

Translated from http://deeplearning.net/tutorial/