Difference between discriminative model and generative model (discriminative model and generative model)

Source: http://blog.csdn.net/amblue/article/details/17023485

In NLP and machine learning, we often encounter these two significantly different models, which have different performances in the Learning (training) and evaluation (TEST) stages.

To sum up the differences between them, please add:

1. The essential difference between the two is that the modeling objects are different.

Suppose there are sample input values (or observed values) x, category tags (or output values) y

The evaluation object of the discriminative model is to maximize the conditional probability P (Y | X) and directly model it. The evaluation object of the generative model is to maximize the joint probability p (x, y) and model it.

In fact, the evaluation goal of both is to obtain the final category tag y, while y = argmax P (Y | X ), the difference is that the discriminant model obtains model parameters directly by solving the optimal problem that satisfies the distribution of training samples. It mainly uses the Laplace Multiplication Algorithm and gradient descent method, common discriminant models, such as the maximum entropy model, CRF, LR, and SVM;

The generative model is first converted to Y = argmax P (Y | X) = argmax p (x | y) * P (y) through Bayes, and then learned P (y) respectively) and P (x | Y) probability distribution, mainly through the maximum likelihood estimation method to learn parameters, such as Ngram, hmm, Naive Bayes.

2. More flexible discriminative Models

It is mainly reflected in feature selection. generative models generally need to add features to Markov chains, while discriminative models are added to any features for combination.

3. guidance training is required for discriminative models. generative models can be trained without guidance.

As mentioned above, generative models mainly use maximum likelihood estimation for parameter learning. The EM algorithm can obtain the local optimal solution of Maximum Likelihood Estimation when the state value is unknown.