Machine learning-multivariable linear regression

"one, multivariable linear regression model"

Multivariate linear regression refers to the case where the input is a multidimensional feature, for example:

It can be seen that the price of a house is determined by four variables (size, number of bedrooms, number of floors, age of home), in order to be able to predict the price of a house under a given condition (four variables) (y), We need to establish the corresponding linear regression model.

Assuming there are n variables, the corresponding multivariate linear regression model is as follows:

Note that x refers to a training sample where each training sample is a (n+1) dimension vector (with additional x0=1)

"Second, cost function"

The cost function for multivariable linear regression is as follows:

where x (i) represents the first sample

"Three, gradient descent method to find the best theta"

The following lists the univariate linear regression gradient descent method (left) and multivariate linear regression gradient descent method (right)

where α is the learning rate.

"Two details of the gradient descent process":
1. Feature Normalization

By looking at the values, note this House sizes is about the number of bedrooms. When features differ by orders of magnitude, first performing feature scaling can make gradient descent converge much more Quickly

That is, when there is a large difference between features, such as the size of the house and the number of bedrooms, this will cause the gradient descent convergence is relatively slow, as shown in (left) , when the characteristics are normal, gradient descent convergence faster, as shown on (right) .

The regularization method can use mean value and the standard deviation method, or other methods.

2. Selecting Learning Rates

The correct choice of learning rate should ensure that the cost function is degraded after each iteration, as shown in:

If the learning rate α is too large, it may cause the cost function to rise instead of always showing a downward trend, such as.

But the learning rate α is too small, and that gradient decline will converge too slowly.

"How to choose the learning rate"

"Four, the best theta of the normal equation"

, J=0,1,............N
By

The

Note When the following conditions occur:

Redundant feature can then be deleted.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Machine learning-multivariable linear regression