Difference between discriminative model and generative model (discriminative model and generative model)

Source: Internet
Author: User

Source: http://blog.csdn.net/amblue/article/details/17023485

In NLP and machine learning, we often encounter these two significantly different models, which have different performances in the Learning (training) and evaluation (TEST) stages.

To sum up the differences between them, please add:

1. The essential difference between the two is that the modeling objects are different.

Suppose there are sample input values (or observed values) x, category tags (or output values) y

The evaluation object of the discriminative model is to maximize the conditional probability P (Y | X) and directly model it. The evaluation object of the generative model is to maximize the joint probability p (x, y) and model it.

In fact, the evaluation goal of both is to obtain the final category tag y, while y = argmax P (Y | X ), the difference is that the discriminant model obtains model parameters directly by solving the optimal problem that satisfies the distribution of training samples. It mainly uses the Laplace Multiplication Algorithm and gradient descent method, common discriminant models, such as the maximum entropy model, CRF, LR, and SVM;

The generative model is first converted to Y = argmax P (Y | X) = argmax p (x | y) * P (y) through Bayes, and then learned P (y) respectively) and P (x | Y) probability distribution, mainly through the maximum likelihood estimation method to learn parameters, such as Ngram, hmm, Naive Bayes.

2. More flexible discriminative Models

It is mainly reflected in feature selection. generative models generally need to add features to Markov chains, while discriminative models are added to any features for combination.

3. guidance training is required for discriminative models. generative models can be trained without guidance.

As mentioned above, generative models mainly use maximum likelihood estimation for parameter learning. The EM algorithm can obtain the local optimal solution of Maximum Likelihood Estimation when the state value is unknown.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.