Linear regression Linear Regression

hypothesis function

Suppose we have a set of house size and price of the relationship between the data, and the area as the horizontal axis, the price as the ordinate to draw it out, with such a data, suppose to give you a house area, how to predict the price of the house?

　　　　　　　　

A straightforward approach is to use training data to fit a straight line out, in linear regression we call hypothesis function, where there is only one eigenvalue (i.e. house price), so hypothesis function is $ $h _{\theta} (x) =\ Theta_{0} x_{0} + \theta_{1} x_{1}$$ where $x_{0}=1$,

When there are multiple eigenvalues, the hypothesis function is: $ $h _{theta} (x) = \sum_{i=0}^n \theta_i x_i = \theta^t x$$

So now given a set of training data, how do we choose $\theta$?

Cost function

In order to solve the above mentioned choice $\theta$ problem, our practice is to make $h (x) $ and $y$ values closest, that is, we do not have a group of data will have a predictive value $h (x) $ (through the hypothesis function solution) and the real value Y, What we need to do is to minimize the difference between the predicted value of all sample data and the real value, so define the following cost function: $ $J (\theta) = \frac{1}{2} \sum_{i=1}^m (H_{\theta} (x^{(i)})-y^{(i)}) ^2$$

This cost function is actually a regression model of least squares. Here is a description of the characteristic value:

The cost function is drawn in the following form:

The solution of Gradient gradient descent method

We want to choose $\theta$ to make the $j (\theta) $ minimum, here we use the gradient descent method to solve. $J (\theta) $ is a function of $\theta$ as a variable, so it is actually the $\theta$, then the gradient is actually a vector consisting of the partial derivative of each variable, the size of the vector represents the value of the function change, the direction of the vector indicates the direction of the function value falling fastest.

The gradient descent method is described as follows:

• Start pick $\theta$ Initial value $ (\tehta_0, \theta_1, ..., \theta_n) ^t$
• Change the value of $theta$ each time to reduce $j (\theta) $, knowing we get a minimum value

$\theta$ Each Update method is as follows: $$\theta_j: = \theta_j-\alpha \frac{\partial}{\partial\theta_j}j (\theta) $$

Here $\alpha$ is the learning rate (learning rates), in order to achieve this algorithm, we need to ask $\frac{\partial}{\partial\theta_j}$:

So the rules for updating are as follows:

Gradient descent in the form of effects:

The last line drawn:

Using MATLAB to achieve gradient descent method

The training sample form is as follows:

The

Linear regression Linear Regression