Probability interpretation of least squares-maximum likelihood method

For a conventional linear model, its precise model can be defined as follows:

The first part of the model describes the trend of the variation with the self-variable, while the second part describes the error term that the linear model cannot be modeled.

The least squares method uses the loss cost and the minimum to obtain the parameters of the linear model. In this paper, we find the model parameter Θ by the assumption of the probability distribution of error term and the method of maximum likelihood estimation .

Here, we assume that the error term ε is independently distributed and conforms to the Gaussian distribution of the mean value of 0, i.e.:

, the expected value is also subject to the Gaussian distribution of the mean value:

The upper formula can be understood as the probability distribution of thepredicted value y when given x, θ, and we can also give the likelihood function of θ:

Because the error items of different observation points obey the independent distribution, the whole likelihood function can be written as:

According to the idea of maximum likelihood estimation, the log maximization of likelihood value can be obtained by:

That is:

?

?

Note: The knowledge of maximum likelihood estimation is used in this paper, and the precondition is given, that is, the error term obeys the independent Gaussian distribution.

Here's a very good article about maximum likelihood estimation. http://blog.csdn.net/yanqingan/article/details/6125812

Probability interpretation of least squares-maximum likelihood method