These two terms are always heard, but they are not clear about their essential differences. Today, I spent a little time figuring it out. The conclusion is as follows:
The Discriminative model is a discriminant model, also known as a conditional model or a conditional probability model.
Generative model is a generative model.
The essential difference between the two is that
The Discriminative model estimates the conditional probability distribution (conditional distribution) P (class | context)
The generative model estimates the joint probability distribution (joint probability distribution) P ()
Common generative models include:
-Gaussians, Naive Bayes, mixtures of Multinomials
-Mixtures of gaussians, mixtures of experts, HMMs
-Sigmoidal belief networks, Bayesian Networks
-Markov Random Fields
Common discriminative models include:
-Logistic Regression
-SVMs
-Traditional Neural Networks
-Nearest Neighbor
Successes of generative methods:
? NLP
-Traditional rule-based or boolean logic systems
Dialog and Lexis-Nexis) are giving way to statistical
Approaches (Markov models and stochastic context
Grammars)
? Medical Diagnosis
-QMR knowledge base, initially a heuristic expert
Systems for reasoning about diseases and symptoms
Been augmented with demo-theoretic formulation
? Genomics and Bioinformatics
-Sequences represented as generative HMMs
The main application is discriminative model:
? Image and document classification
? Biosequence Analysis
? Time Series Prediction
Discriminative model disadvantages:
? Lack elegance of generative
-Priors, structure, uncertainty
? Alternative notions of penalty functions,
Regularization, kernel functions
? Feel like black-boxes
-Relationships between variables are not explicit
And visualizable
Bridging generative and discriminative:
? Can performance of SVMs be combined
Elegantly with flexible Bayesian statistics?
? Maximum Entropy discrimination marries
Both methods
-Solve over a distribution of parameters (
Distribution over solutions)
From http://billlangjun.blogspot.com/2006/09/discriminative-vs-generative-model.html