Linear model (i)--Introduction to Generalized linear model (GLM)

We begin to contact linear equations from junior middle school, and linearity is the simplest relationship between variables, so I intend to start with the linear model to introduce the basic algorithm of machine learning. Generalized linear model (General Linear MODEL,GLM) is a generalization form of linear regression model, which can be deduced from linear regression, logistic regression, Softmax regression and so on. Most of the books that I have seen before, including the most recognized Standford cs229 courses, are the first to introduce specific regression models, and then to abstract the generalized linear regression model, when I follow such a learning route in the study of specific regression model, it is not very important to understand why some knowledge points to use, For example, the logistic regression of the sigmoid function, the initial contact when originally thought is artificially defined, not too much to pursue its essential source, just in the mathematical form know how it is deduced. But in the later generalized linear model, I understand the origin of the sigmoid function. I have always thought that mastering the formula derivation only understand the model expression, only to understand the source of the model is really mastered the model itself, so personally think before learning linear model, it is necessary to understand the form of a generalized linear model, so that in learning the specific regression model, help to understand the model itself.

The generalized linear model is based on the exponential distribution family, and the mathematical model of the exponential distribution family is as follows:

The natural parameter, which can be a vector, is called the sufficient statistic, and can also be used as a vector, in general.

The generalized linear model constructed from an exponential distribution family is based on the following three assumptions:

1, given the characteristic attribute x and the parameter, the conditional probability of y obeys the exponential distribution family, namely

2, the forecast of the expectation, that is, the calculation

3, and the x is linear, that is,

The above is the definition of the generalized linear model, we do not need to delve into this model at present, after the specific linear regression and logistic regression, I will introduce the relationship between these models and the generalized linear model in detail.