"Reprinted" Discriminative Learning and generative learning

discriminative Learning and generative learning2011-12-08 10:47 1929 people read comments (2) favorite reports Variablesdependencies algorithm Includeparametersexpress

Discriminative Learning algorithm is a kind of model input (X) output (Y) of the method of the relationship, simply like Chinese medicine, we only know with a number of drugs (Angelica, tiger bone ...) Can be made into a prescription, you can treat the pain. But we do not understand the internal reasons, we see it as a black box, just understand the relationship between X and Y, that is, model P (y|x). The result of learning is a set of parameters of X, like Angelica 2 money, tiger bone 12. Discriminative learning requires that the individual items of x be independently changed.

Generative learning algorithm is not directly model P (y| X), because this can be very complex, such as model in give a x= smoker, Y has the chance of lung cancer. But model P (x| Y), it's like looking at smokers in people with lung cancer. We all know that smokers Shing, but people with lung cancer approximation is a few, model from the lung cancer people start, more easily, such as 100,000 lung cancer, sampling 1000 is almost.

The generative algorithm needs to examine the probability distribution of each item in x, i.e. P (y|{ X1,X2,X3...XN}), you need to investigate P (x1| Y) ... P (xn| Y) ...

In a word, the direct model P (y| X) algorithm is discriminative Learning,model P (x| y) and P (Y) and then the indirect model P (y| X) is the generative learning.

The following is from Wikipedia

Http://en.wikipedia.org/wiki/Discriminative_model

Discriminative model

From Wikipedia, the free encyclopedia

discriminative models  are a class of Models used In machine Learning for modeling the dependence of an unobserved variable  y  on an observed variable  x . Within a statistical framework , this was done by modeling the conditional probability Distribution  P ( y  |  x ), which can is used for Predicting  y  from  x .

discriminative models differ from generative models  in that they does not allow one to generate samples from The joint distribution of  x  and  y . However, for tasks such as classification and regression that does not require the joint distribution, Discriminative models generally yield superior performance. On the other hand, generative models is typically more flexible than discriminative models in expressing dependencies in Complex learning tasks. In addition, most discriminative models is Inherently supervised and cannot easily be extended to unsuper Vised learning.

Examples of discriminative models used in machine learning include:

• Logistic regression, a type of generalized linear regression used for predicting binary or categorical outputs (also known As maximum entropy classifiers)
• Linear discriminant Analysis
• Support Vector Machines
• Boosting
• Conditional Random fields
• Linear regression
• Neural networks

Generative model

From Wikipedia, the free encyclopedia

In probability and statistics, a generative model was a model for randomly generating observable data, typically g Iven some hidden parameters. IT specifies a joint probability distribution over observation and label sequences. Generative models is used in machine learning for either modeling data directly (i.e., modeling observed draws from a pro Bability density function), or as an intermediate step to forming aconditional probability density function. A conditional distribution can is formed from a generative model through the use of the Bayes ' rule.

shannon  (1948) gives an example in which a table of Frequencies of Chinese word pairs is used to generate a sentence beginning with "representing and speedily are an good"; Which is not proper 中文版 but which would increasingly approximate it as the table is moved from word pairs to Word tripl ETS etc.

generative models contrast with discriminative models , in the A generative model is a full probabilistic model of all variables, whereas a discriminative model provides a mod El only for the target variable (s) conditional on the observed variables. Thus a generative model can be used, for example, to simulate (i.e.  generate ) values of all variable in the Model, whereas a discriminative model allows only sampling of the target variables conditional on the observed quantities. On the other hand, despite the fact that discriminative models does not need to model the distribution of the observed Vari Ables, they cannot generally express more complex relationships between the observed and target variables. They don ' t necessarily perform better than generative models at classification and regression  Tasks.

Examples of generative models include:

• Gaussian mixture model and other types of mixture model
• Hidden Markov Model
• Naive Bayes
• Aode
• Latent Dirichlet allocation
• Restricted Boltzmann Machine

If the observed data is truly sampled from the generative model and then fitting the parameters of the generative model to M Aximize The data likelihood is a common method. However, since most statistical models is only approximations to the true distribution, if the model ' s Applicati On are to infer about a subset of variables conditional on known values of others, then it can be argued that the Approxima tion makes more assumptions than is necessary to solve the problem at hand.  In such cases, it's often more accurate to model the conditional density functions directly, using a discriminative model (see above).

"Reprinted" Discriminative Learning and generative learning