Discriminative model and generative model

Discriminative Model

This model mainly models p (y | x) and uses x to predict y. During modeling, you do not need to pay attention to the joint probability distribution. Only focus on how to optimize p (y | x) to split the data. Generally, discriminative models are better than generative models in classification tasks. However, the process of discriminative model modeling is usually supervised and cannot be expanded to unsupervised.

Common discriminant models include:

Logistic regression

Linear discriminant analysis

Support vector machines

Boosting

Conditional random fields

Linear regression

Neural networks

Generative model

This model models the joint probability distribution p (x, y) of the observed sequence. After obtaining the joint probability distribution, conditional probability distribution can be obtained through Bayesian formula. Generative models carry more information than discriminative models. In addition, generative models are easy to implement incremental learning.

Common generative models include:

Gaussian mixture model and other types of mixture model

Hidden Markov model

Naive Bayes

AODE

Latent Dirichlet allocation

Restricted Boltzmann Machine

As we can see from the above, the most important difference between the discriminative model and the generative model is that the objective during training is different. The Discriminative model mainly optimizes the conditional probability distribution to make the x and y correspond more, in classification, it is more severable. The model is mainly used to optimize the joint distribution probability of training data. Meanwhile, the generated model can be obtained through Bayesian, but the generated model cannot be obtained.