"Machine Learning Classic algorithm Source Analysis series"--Linear regression

One, single variable linear regression:

1. Data Set Visualization

2. Solving model parameters

For linear regression models, there are two ways to solve model parameters.

1) Gradient Descent method

Take the cost function into the expansion:

MATLAB Code implementation:

2) Normal equation

MATLAB Code implementation:

On the derivation of the normal equation:

3) Comparison of gradient descent method and normal equation

From the console output model parameters and regression lines, the results are very similar, the specific choice of what method to reference the following criteria.

Linear regression of multivariate variables

The multivariate linear regression solution is similar to the one-variable linear regression, and one of the notable problems is:

For more than one characteristic variable, we should remember to do feature scaling before calculating the model parameters, so that the characteristic values are roughly in the same range, which is advantageous to the gradient descent algorithm to solve the model parameters.

Describes one of the following methods:

MATLAB implementation:

In comparison, the normal equations omit the step of feature scaling when dealing with multivariable regression equations, simply follow the steps of a single variable and be more concise.

Three, the choice of learning rate

The efficiency of gradient descent is greatly influenced by the learning rate, which is too small, the convergence rate is very slow, and the number of iterations is increased; when too large, each iteration may not reduce the cost function, or even cross the local minimum value cannot converge.

Options that can be made by means of a test: such as 0.01,0.03,0.1,0.3,1,3,10 ...

PS: Source code interpretation for the Stanford University Machine Learning Program (2014 edition)-the industry Daniel Andrew Ng (Wu Enmin) as the keynote speaker.

PPS: This series of course notes "Brother Pure Handwriting":http://www.cnblogs.com/always-chang/p/5935787.html

Video URL:https://www.coursera.org/learn/machine-learning

Video download:http://pan.baidu.com/s/1dEJcXrj Password: bnks

Source Detailed analysis:http://pan.baidu.com/s/1bo8rIXL Password: hdn7

"Machine Learning Classic algorithm Source Analysis series"--Linear regression