Machine Learning:linear Regression with multiple Variables

Machine Learning:linear Regression with multiple Variables

Then the last example of predicting the price of a house leads to a multivariable linear regression.

Here we use the representation of vectors to make the expression more concise.

variable gradient descent as with a single variable, all theta values need to be updated synchronously.

The reason for feature scaling is to speed up the convergence of the gradient descent algorithm. As shown, the left figure Theta2 and theta1 The magnitude difference is too large, so that the cost function of the contour graph is a high-oval shape, you can see its convergence trajectory, so too slow! The image on the right is a feature scaling, and the theta range is similar, which accelerates the convergence speed.

Feature scaling does not have a strict guideline, just to make the values of each feature range similar.

A feature scaling method is presented here, x (i) = (x (i)-average (x))/(Max-min). Of course, other methods can be as long as the scope of the feature is reasonable.

Also , when we feature scaling the training data in the training set, we need to do the same for a new example when we make predictions, That means we need to save some data such as average (X), max,min, etc.

Let's repeat the learning rate selection question.

The choice of polynomial order is also an issue to consider, as it is related to the problem of under-fitting (not fitting) and over-fitting (overfitting), which we will discuss in the subsequent post.

Here we introduce a method that does not require many iterations to find the optimal theta: normal equation (normal equation).

A proof of J (Theta) process is attached below.

then make j (theta) = 0, you can get the above formula.

The above content compares the gradient descent and the advantages and disadvantages of the normal equation.

If X ' *x is irreversible (i.e. | X ' *x|=0), so what should we do with it?

----------------------------------------------------------------------------------------------------------- -------

This content is from Stanford University Andrew Ng teacher "machine learning" courseware

Machine Learning:linear Regression with multiple Variables