Basic outline of machine learning

Basic mathematics (2 courses)

Calculus

Limit, E, derivative, differential, integral

Partial Derivative, direction derivative, gradient

Extreme Value, multivariate function extreme value, multivariate function Taylor expansion

Unlimited optimization and Constrained Optimization

Multiplier, a dual problem

 

Linear Algebra

Matrix, determinant, Elementary Transformation

Linear correlation, linear independence

Rank, feature value, feature vector

Orthogonal vector and orthogonal matrix

Matrix decomposition

 

Probability

Random Variables, probability density functions, Distribution Functions

Conditional probability, full probability formula, Bayesian Formula

Expectation, variance

Big Number Theorem and central limit theorem

Covariance, correlation coefficient

Common probability distribution and Poisson distribution

Exponential Family distribution, multivariate Gaussian distribution

Parameter Estimation, moment estimation, maximum likelihood estimation MLE

 

Basic concepts of machine learning (course 1)

Input space, feature space, and output space

Joint probability distribution, hypothesis Space

Three elements: method = model + strategy + Algorithm

Loss function, risk function, empirical risk, and structural risk

MLE and map

 

Perceptron (class 1)

Sensor model, learning strategy, Training Method

0-1 loss function

Geometric interpretation of the Sensor

Proof of perception Machine

Pocket perceptron

 

Linear regression and logistic regression (2 lessons)

Loss function, training method, geometric interpretation, square loss function

Gradient Descent

Logistic regression form, derivation and training, and logistic loss

Quasi-Newton method, lbfgs

 

Machine Learning diagnosis and debugging (1 course)

Training error, test error, underfitting, overfitting

Normalization and cross-validation

 

Tree Model and boost (3 lessons)

Entropy definition and application, information gain

Decision tree, ID3, C4.5, and cart

Adaboost, exponential loss function

Gbdt

Random forest random Forest

 

SVM (3-4 lessons)

Maximum hard interval, function interval, and geometric Interval

Maximum soft Interval

Dual Algorithm

Page loss function

Core functions and Techniques

SMO Algorithm

 

Maximum Entropy model (1 course)

Model Definition, constraints, and Derivation

A new understanding of Logistic Regression

 

Neural Network (1 course)

Model Definition and training

BPA Algorithm

 

Unsupervised learning (3 courses)

K-means and Gaussian mixture model GMM

EM algorithm, derivation, interpretation, and understanding

Topic model basics, SVD, lsa, plsa, lda

 

Summary (1 course)

Loss Function Comparison

Comparison and Selection of Models

General steps for solving actual problems

 

Basic outline of machine learning