Mathematics behind SVM

Inner Product of a vector)

For Vectors

\ [\ Begin {array} {L}
U = \ left [{\ begin {array} {* {20} {c }}
{U_1 }}\\
{U_2 }}
\ End {array }}\ right] \
V = \ left [{\ begin {array} {* {20} {c }}
{V_1 }}\\
{V_2 }}
\ End {array }}\ right]
\ End {array} \]

Their Inner Product

\ [{U ^ t} v = {U_1} {v_1} + {u_2} {V_2} \]

Youyou

\ [\ Left \ | U \ right \ | =\ SQRT {U_1 ^ 2 + u_2 ^ 2} \]

Assume that the projection distance of V on U is P

\ [{U ^ t} v = p \ left \ | U \ right \ | \]

If the angle between U and V is greater than 90 °, p is the plural, and 90 ° is the positive number.

With the above foundation, we can see that the loss of SVM is a function.

\ [\ Underbrace {\ min} _ \ Theta \ left \ {c \ left [{\ sum \ limits _ {I = 1} ^ m {y ^ {\ left (I \ right )}} {\ mathop {\ RM cos} \ NoLIMITS} {T_1} \ left ({\ Theta ^ t} {x ^ {\ left (I \ right )}}} \ right) + \ left ({1-{y ^ {\ left (I \ right) }}\ right) {\ mathop {\ RM cos} \ NoLIMITS} {T_0} \ left ({\ Theta ^ t} {x ^ {\ left (I \ right )}}} \ right )}} \ right] + \ frac {1} {2} \ sum \ limits _ {j = 1} ^ n {\ Theta _ j ^ 2 }}\ right \} \]

Support Vector Machine tries to minimize this formula, for the first half

\ [{C \ left [{\ sum \ limits _ {I = 1} ^ m {y ^ {\ left (I \ right )}} {\ mathop {\ RM cos} \ NoLIMITS} {T_1} \ left ({\ Theta ^ t} {x ^ {\ left (I \ right )}}} \ right) + \ left ({1-{y ^ {\ left (I \ right) }}\ right) {\ mathop {\ RM cos} \ NoLIMITS} {T_0} \ left ({\ Theta ^ t} {x ^ {\ left (I \ right )}}} \ right) }}\ right]} \]

When y = 1, θ TX will optimize θ toward the trend of θ TX ≥ 1

When y = 0, θ TX will optimize θ toward the trend of θ TX ≤-1

For the second half

\ [{\ Frac {1} {2} \ sum \ limits _ {j = 1} ^ n {\ Theta _ j ^ 2} \]

For the following two cases

 

For ease of understanding, here we can simplify θ with only two parameters (θ 1, θ 2), and θ 0 = 0, then

\ [\ Frac {1} {2} \ sum \ limits _ {j = 1} ^ n {\ Theta _ j ^ 2 }=\ frac {1} {2 }\ left ({\ Theta _ 1 ^ 2 + \ Theta _ 2 ^ 2} \ right) = \ frac {1} {2} {\ left ({\ SQRT {\ Theta _ 1 ^ 2 + \ Theta _ 2 ^ 2} \ right) ^ 2 }=\ frac {1} {2} {\ left \ | \ Theta \ right \ | ^ 2} \]

We can see that SVM tries to minimize θ to minimize this part.

Combined with the first part

When y = 1, θ TX will optimize θ toward the trend of θ TX ≥ 1, and The θ of the second part should be as small as possible, then

\ [{\ Theta ^ t} {x ^ {\ left (I \ right) }={ P ^ {\ left (I \ right )}} \ left \ | \ Theta \ right \ | \]

P (I) needs to be as big as possible

 

Support Vector opportunity selects the latter because it has a relatively large P.

Mathematics behind SVM