"Reprint" UFLDL Tutorial (the main ideas of unsupervised Feature learning and deep learning)

UFLDL tutorialfrom ufldl Jump to:navigation, search

Description:  This tutorial would teach you the main ideas of unsupervised Feature learning and deep learning. By working through it, you'll also get to implement several feature learning/deep learning algorithms, get to see them w Ork for yourself, and learn how to apply/adapt these ideas to new problems.

This tutorial assumes a basic knowledge of machine learning (specifically, familiarity with the ideas of supervised Learni  Ng, logistic regression, gradient descent). If you is not familiar with these ideas, we suggest the go to this machine learning course and complete sections II, III, IV (up to Logistic Regression) first.

Sparse Autoencoder

• Neural Networks
• BackPropagation algorithm
• Gradient Checking and advanced optimization
• Autoencoders and Sparsity
• Visualizing a trained Autoencoder
• Sparse Autoencoder Notation Summary
• Exercise:sparse Autoencoder

vectorized implementation

• Vectorization
• Logistic Regression vectorization Example
• Neural Network vectorization
• Exercise:vectorization

PREPROCESSING:PCA and Whitening

• Pca
• Whitening
• Implementing pca/whitening
• EXERCISE:PCA in 2D
• EXERCISE:PCA and Whitening

Softmax Regression

• Softmax Regression
• Exercise:softmax Regression

Self-taught learning and unsupervised Feature learning

• Self-taught learning
• Exercise:self-taught Learning

Building Deep Networks for classification

• From self-taught learning to deep Networks
• Deep Networks:overview
• Stacked Autoencoders
• Fine-tuning stacked AEs
• Exercise:implement deep networks for digit classification

Linear decoders with Autoencoders

• Linear Decoders
• Exercise:learning color features with Sparse autoencoders

Working with Large Images

• Feature extraction using convolution
• Pooling
• Exercise:convolution and Pooling

Note: The sections above this line is stable.  The sections below is still under construction, and may change without notice. Feel free to browse around however, and feedback/suggestions is welcome.

Miscellaneous

• MATLAB Modules
• Style Guide
• Useful Links

Miscellaneous Topics

• Data preprocessing
• Deriving gradients using the backpropagation idea

Advanced Topics:

Sparse Coding

• Sparse Coding
• Sparse Coding:autoencoder Interpretation
• Exercise:sparse Coding

ICA Style Models

• Independent Component Analysis
• Exercise:independent Component Analysis

Others

• convolutional Training
• Restricted Boltzmann Machines
• Deep belief Networks
• Denoising autoencoders
• K-means
• Spatial Pyramids/multiscale
• Slow Feature Analysis
• Tiled convolution Networks

Material contributed by:andrew Ng, Jiquan Ngiam, Chuan Yu Foo, Yifan Mai, Caroline Suen

"Reprint" UFLDL Tutorial (the main ideas of unsupervised Feature learning and deep learning)