Machine learning--polynomial regression and normal equation

Source: Internet
Author: User

The last time we shared a multivariate linear regression model (Linear Regression with multiple Variables), let's talk about polynomial regression (polynomial Regression) and the normal equation (normal equation). (we still discuss the problem of house price forecast)

Polynomial regression

Sometimes, linear regression does not apply to all of the data, we need curves to fit our data, such as a two-square model:

Or a three-cubic-square Model:

The two models we plotted in the coordinate system are as follows:

In general, we need to look at the data before deciding what model to use to deal with the problem.

In addition, we can also make:

This transforms the higher order equation model into a linear regression model. This is also a feature scaling (Features scaling) .

PS: If our model uses a polynomial regression model, feature scaling is necessary before the gradient descent algorithm is run.

Normal equation

So far, the machine learning algorithms we have studied are all gradient descent algorithms (grandient descent). But for some regression problems, there is a better solution to the normal equation.

The normal equation is to find the parameter that minimizes the cost function by solving the following equation:

Assuming our training feature matrix is x (which contains x0=1) and our training set results in Vector y, the normal equation is used to solve the vector:

Callout: T denotes transpose of matrix X, 1 indicates inverse of matrix X

We use the price forecast problem data:

The data includes four features (excluding X0), we add x0=1, and we use the normal equation method to solve:

In matlab, normal equation writing: PINV (x ' * x) *x ' *y

There is a need to be aware that there are some irreversible matrices (usually because the features are not independent, such as the size of both feet and the size of the meter, it is possible that the number of features is greater than the number of training sets, such as 2000 features but only 1000 training sets), The normal equation method is not able to be used.

So we now have two machine learning algorithms, one is the gradient drop, one is the normal equation, two methods are compared as follows:

At this point, the question of linear regression is discussed here. In the next phase we will discuss logistic Regression.

Machine learning--polynomial regression and normal equation

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.