Machine Learning Study Notes

See Professor Max Welling on the home page there are a lot of learning notes, a collection of it, its recently published a book it has not yet seen.

Http://www.ics.uci.edu/~welling/classnotes/classnotes.html

Statistical Estimation [PS]
-Bayesian estimation
-Maximum a posteriori (MAP) estimation
-Maximum likelihood (ML) estimation
-Bias/variance Tradeoff & Minimum description length (MDL)

expectation maximization (EM) algorithm [PS]
- Detailed derivation plus some examples

supervised Learning (Function approximation) [ps]
-Mixture of experts (MoE)
-Cluster weighted modeling (CWM)

Clustering [PS]
-Mixture of Gaussians (MoG)
-Vector Quantization (VQ) with K-means.

Linear Models [PS]
-Factor Analysis (FA)
-Probabilistic principal component analysis (PPCA)
-Principal component Analysis (PCA)

Independent Component Analysis (ICA) [PS]
-Noiseless ICA
-Noisy ICA
-Variational ICA

Mixture of Factor analysers (MoFA) [PS]
-Derivation of learning algorithm

Hidden Markov Models (HMM) [PS]
-Viterbi decoding algorithm
-Baum-welch Learning Algorithm

Kalman Filters (KF) [PS]
-Kalman Filter algorithm (very detailed derivation)
-Kalman Smoother algorithm (very detailed derivation)

approximate inference algorithms [PS]
-Variational EM
-Laplace approximation
-Importance sampling
-Rejection sampling
-Markov chain Monte Carlo (MCMC) sampling
-Gibbs Sampling
-Hybrid Monte Carlo sampling (HMC)

Belief Propagation (BP) [PS]
-Introduction to BP and gbp:powerpoint presentation [PPT]
-Converting directed acyclic graphical models (DAG) into junction trees (JT)
-Shafer-shenoy belief propagation on junction trees
-Some examples

Boltzmann Machine (BM) [PS]
-Derivation of learning algorithm

generative topographic Mapping (GTM) [PS]
-Derivation of learning algorithm

Introduction to Kernel methods:powerpoint presentation [ppt]

Kernel Principal Components analysis [PDF]

Kernel Canonical Correlation analysis [PDF]

Kernel support Vector machines [PDF]

Kernel Ridge-regression [PDF]

Kernel support Vector Regression [PDF]

Convex optimization [PDF]
A Brief introduction based on Stephan Boyd's book, Chapter 5.

Fisher Linear discriminant analysis [PDF]

Machine Learning Study Notes