Discriminant model and generation model

Summary
-Generate model: infinite sample = = "probability density model = generation model = =" Prediction
-Discriminant Model: finite sample = = "discriminant function = Predictive model = =" Prediction

Introduction
Simply put, suppose O is the observed value, Q is the model.
If P (o|q) is modeled, it is the generative model. The basic idea is to establish the probability density model of the sample, and then use the model to predict the inference. Requires a known sample infinity or as large a limit as possible.
This method is generally based on statistical mechanics and Bayes theory.
If the conditional probability (posterior probability) P (q|o) is modeled, it is the discrminative model. The basic idea is to establish discriminant function under the condition of finite sample, not to consider the model of sample generation, and to study the prediction model directly. The representative theory is the statistical learning theory.
The two methods are now more intersecting.

"Discriminant models discriminative Model"--inter-class Probabilistic description

It can also be called a conditional model, or a conditional probability model. It is estimated that the conditional probability distribution (conditional distribution), p (class|context).
Using positive and negative examples and classification labels, focus is used to discriminate the edge distribution of the model. The objective function directly corresponds to the classification accuracy rate.

-Main Features:
Finding the optimal classification surface between different categories reflects the difference between heterogeneous data.
Advantages
Classification boundaries are more flexible and more advanced than using pure probabilistic methods or production models.
Can clearly distinguish between multiple classes or a class of different characteristics from other classes
In clustering, viewpoint changes, partial occlusion and scale variations, the effect is better
Suitable for more categories of identification
The performance of discriminant model is simpler and more easy to learn than the model.
Disadvantages
Does not reflect the nature of the training data itself. Limited ability to tell you whether it is 1 or 2, but there is no way to describe the whole scene.
Lack Elegance of generative:priors, structure, uncertainty
Alternative notions of penalty functions, regularization, nuclear function
Black box operation: The relationship between variables is unclear, not visible

-Common mainly include:
Logistic regression
SVMs
Traditional neural networks
Nearest Neighbor
Conditional random Fields (CRF): The latest hot models, generated from the NLP field, are evolving to ASR and CV.

-Main applications:
Image and document classification
Biosequence Analysis
Time series prediction

"Generate model Generative models"--intra-class Probabilistic description

Also known as the production model. It is estimated that the joint probability distribution (joint probability distribution), p (class, context) =p (Class|context) *p (context).

The observation values are modeled for random generation, especially given the case of certain hidden parameters. In machine learning, or for modeling data directly (using the probability density function to model the observed draw), or as an intermediate step in generating conditional probability density functions. Conditional distributions can be obtained from the generation model by using Bayesian rule.

If the observed data is generated entirely by the build model, you can fitting the parameters of the model to generate only possible data similarity. However, the data can rarely be fully generated by the generation model, so the more accurate way is to directly model the conditional density function, that is, using classification or regression analysis.

Unlike the description model, all variables in the description model are measured directly.

-Main Features:
In general, it is the model of posterior probability, which represents the distribution of data from the angle of statistics, and can reflect the similarity of similar data.
Focus only on the Inclass itself (that is, the probability in the lower-left corner of the point) and don't care where the decision boundary is.
Advantages
Actually, the information is richer than the discriminant model,
The study of single-class problem is more flexible than discriminant model
Models can be obtained by incremental learning
Can be used for data incomplete (missing) situations
Modular construction of composed solutions to complex problems
Prior knowledge can be easily taken to account
Robust to partial occlusion and viewpoint changes
Can tolerate significant intra-class variation of object appearance
Disadvantages
tend to produce a significant number of false positives. This is particularly true to object classes which share a high visual similarity such as horses and cows
The learning and computing process is more complex

-Common mainly include:
Gaussians, Naive Bayes, mixtures of multinomials
Mixtures of Gaussians, mixtures of experts, HMMs
Sigmoidal belief networks, Bayesian networks
Markov Random fields

The generative model listed can also be trained using the Disriminative method, such as GMM or Hmm, the training method has EBW (Extended Baum Welch), or the FEI margin method proposed by large Sha recently.

-Main applications:
Nlp:
Traditional rule-based or Boolean logic systems (Dialog and Lexis-nexis) are giving-to statistical approaches (Markov Models and stochastic context grammars)
Medical Diagnosis:
QMR Knowledge Base, initially a heuristic expert systems for reasoning on diseases and symptoms been augmented with Dec Ision theoretic formulation Genomics and bioinformatics
Sequences represented as generative HMMs

"The relationship between the two"
The discriminant model can be obtained from the generation model, but the model cannot be generated by the discriminant model.
Can performance of SVMs is combined elegantly with flexible Bayesian statistics?
Maximum Entropy Discrimination marries both methods:solve over a distribution of parameters (a distribution over solution S

"Reference url"
Http://prfans.com/forum/viewthread.php?tid=80
Http://hi.baidu.com/cat_ng/blog/item/5e59c3cea730270593457e1d.html
Http://en.wikipedia.org/wiki/Generative_model
Http://blog.csdn.net/yangleecool/archive/2009/04/05/4051029.aspx

==================
Comparison of three models: HMMs and MRF and CRF

Http://blog.sina.com.cn/s/blog_4cdaefce010082rm.html

HMMs (Hidden Markov model):
The state sequence cannot be observed directly (hidden);
Each observation is considered to be a random function of the state sequence;
The state transition matrix is a random function that changes the state according to the transfer probability matrix.
The difference between HMMs and MRF is that it contains only the label field variables, not the observed field variables.

MRF (Markov Random Airport)
Simulates an image into a grid of random variables.
Each of these variables has a definite dependency (Markov nature) of the nearest neighbor, which is composed of random variables outside of itself.

CRF (conditional random field), also known as Markov random domain
A conditional probabilistic model for labeling and slicing ordered data.
Formally, CRF can be regarded as a non-graph model, which examines the conditional probabilities of a given input sequence's labeling sequence.

In the application of vision problems:
HMMs: Image denoising, image texture segmentation, fuzzy image restoration, texture image retrieval, automatic target recognition, etc.
MRF: Image restoration, image segmentation, edge detection, texture analysis, target matching and recognition
CRF: Target detection, recognition, target segmentation in sequential images

P.S.
The marking field is an implicit airport, it describes the local correlation property of pixels, and the model should be based on people's understanding of the structure and characteristics of the image, which has considerable flexibility.
The prior model of spatial domain marking field includes non-causal Markov model and causal Markov model.

This article comes from the Chinese Academy Shong Blog

Discriminant model and generation model