Support Vector Machine (3): The beauty of Soft Margin balance

Many of the materials above preach "the reason why the introduction of soft margin is because the data is not linear", personally think that some errors, in fact, it is difficult to decompose the data, if we use a very complex curved curve to do, or can be decomposed, and mapped to a high-dimensional space is considered to be linear. But if we think about it, there are a lot of algorithms that have the same claim: to look for a flat cross that "maximizes the training set" and "obtains better inductive ability", that is, the so-called overfitting and underfitting. Also like a person's character, too tangled in detail or nerves too big, it is difficult to get along with people happy. We introduced the concept of soft margin when the data for our training set had to be segmented with very complex curves.

In SVM without introducing soft margin, we want the data points in each training set to meet at least the following conditions, that is, the function distance from margin is greater than 0, that is, the distance hyperplane function distance is greater than 1

And considering that if the function distance of some outliers points is less than our expectation, the deviation is ξ, then these points satisfy the condition:

So, we put the previous optimization problem as follows:

Conversions to:

That is to say, on the one hand we need to optimize Ω, making margin=1/| | Ω| | The value is maximized, and on the other hand we choose ω to make the outliers of the deviation of the sum of the smallest, in between the two to seek a balance. C is the balance factor used to adjust the weights between the two-part adjustment items. The optimized Lagrangian function is:

After seeking duality, using KKT condition:

With the return to the original L function, the coefficients of theξ become C-α-r=0 and thus are eliminated, so the dual problem becomes:

As you can see, the form is almost the same as the original problem, and the knowledge adds an upper limit of C to the α condition.

Support Vector Machine (3): The beauty of Soft Margin balance