Linear regression--cost function

Training Set

Training set

Size in Feet2 (x)	Price in + ' s (y)
2104	460
1416	232
1534	315
852	178

Hypothesis:

\[{h_\theta}\left (x \right) = {\theta _0} + \theta {x}\]

Notation:

Θi ' S:parameters

Θi ' s: Parameters

How to Chooseθi ' s?

How to choose θi ' s?

Idea:chooseθ0,θ1so that H (x) was close to Y for our training examples (x, y)

Thought: For the training sample (x, y), select θ0,θ1 to make h (x) close to Y.

Minimize (θ0,θ1) \[\sum\limits_{i = 1}^m {{\left H_\theta ({{}\left (i)}}} x^{)-{\right}} y^i) \right}} \]

Select the appropriate (θ0,θ1) to make \[\sum\limits_{i = 1}^m {{\left ({{H_\theta (i)}}}}\left)-{x^{}}} \right) the minimum.

In order to make the formula more mathematical, change the formula to \[\frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left} {{H_\theta}\left ({{x^{(i)}}} \right)-{y^i}} \right )}^2}} \]

This does not affect the value of (θ0,θ1).

Define cost function \[j\left ({{\theta _0},{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{\left} ({{h_\the Ta}\left ({{x^{(i)}}} \right)-{y^i}} \right)}^2}} \]

Target is \[\mathop {\min imize}\limits_{{\theta _0},{\theta _1}} j\left ({{\theta _0},{\theta _1}} \right) \]

This cost function is also called the squared error cost function (squared error function)

Summarize:

Hypothesis: \[{h_\theta}\left (x \right) = {\theta _0} + \theta {x}\]

Parameters: (θ0,θ1)

Cost Functions: \[j\left ({{\theta _0},{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{\left} ({{H_\theta}\l EFT ({{x^{(i)}}} \right)-{y^i}} \right)}^2}} \]

Goal: \[\mathop {\min imize}\limits_{{\theta _0},{\theta _1}} j\left ({{\theta _0},{\theta _1}} \right) \]

Examples to help understand

First make θ0=0, the cost function becomes \[j\left ({{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left H_\theta ({{}\left ( i)}}} \right)-{y^i}} \right)}^2}} \]

hθ (x)	J (θ1)
For a given θ1, it's a function of X	It's a θ1 function.

Three training samples

X	Y
1	1
2	2
3	3

When Θ1=1, \[j\left ({{\theta _1}} \right) = \frac{1}{{2m}}\left ({{0^2} + {0^2} + {0^2}} \right) = 0\]

When θ1=0.5, \[j\left ({0.5} \right) = \frac{1}{{2*3}}\left ({{{} \left ({0.5-1} \right)}^2} + {{\left}} \rm{1-2) }^2} + {{\left ({{\rm{1}}{\rm{.5-3}} \right)}^2}} \right) \approx {\rm{0}}{\rm{.58}}\]

θ1 values of different values J (θ1)

Each of the different θ1 corresponds to a straight line, our aim is to find the most suitable θ1 (the most suitable straight line)

Linear regression--cost function