Generalized Linear Model

The Linear regression model and the logistic regression model have been reviewed recently, but some of these questions are puzzling, knowing that I see a generalized linear model that is generalized Linear Model before it dawned on the original models are deduced, here to share with you, specific more details can refer to Andrew Ng's course.

First, the exponential distribution

Generalized linear models are derived from exponential distribution, so before you introduce GLM, let's talk about exponential distribution. The exponential distribution is in the following form:

　　　　　　　　　　　　　　　　　

η is a parameter, T (y) is a sufficient statistic of y, i.e. T (Y) can fully express Y, usually T (y) =y. When the parameter t,b,a is fixed, a parameter cluster with the η parameter is defined. In fact, many of the probability distributions are exponential distributions, such as:

(1) Bernoulli distribution

(2) Normal distribution

(3) Poisson distribution

(4) Gamma distribution

And so on .....

Perhaps from the original form can not see that they are exponential distribution, but after a series of transformations, they will find that they are exponential distribution. Give me two examples, and by the way I deduce it myself.

Bernoulli distribution:

　　

So B (y) =1,t (y) =y,η=log (φ/(1-φ)), A (η) =log ((1-φ)), φ=1/(1+e-y), this is the origin of the sigmoid function.

We also change the normal distribution, but here we have to assume that the variance is 1, that the variance does not affect our regression.

　　

We can see Η=µ.

　　

Second, generalized linear model

　　After introducing the exponential distribution, we can see how the generalized linear model is.

First of all, the generalized linear model has three assumptions, and these three assumptions are the prerequisites and the key to help us construct the model.

(1) P (y|x;θ) ~expfamliy (η);

(2) for a given x, our objective function is H (x) =e[t (y) |x];

(3) Η=θtx

Based on the above three assumptions, we can derive the logistic model and the least squares models. The Logistic model is deduced as follows:

H (x) =e[t (y) |x]=e[y|x]=φ=1/(1+e-η) =1/(1+E-ΘTX)

For the least squares model derivation is as follows:

H (x) =e[t (y) |x]=e[y|x]=η=µ=θtx

From this we will refer to the function of the η and the original model parameters called the regular response function. So for the generalized linear model, we need the distribution of Y, we can deduce the corresponding model. Interested in the Softmax regression can be deduced from the polynomial distribution test.

　　

Generalized Linear Model