Support Vector Machine (3): The beauty of Soft Margin balance

Source: Internet
Author: User

Many of the materials above preach "the reason why the introduction of soft margin is because the data is not linear", personally think that some errors, in fact, it is difficult to decompose the data, if we use a very complex curved curve to do, or can be decomposed, and mapped to a high-dimensional space is considered to be linear. But if we think about it, there are a lot of algorithms that have the same claim: to look for a flat cross that "maximizes the training set" and "obtains better inductive ability", that is, the so-called overfitting and underfitting. Also like a person's character, too tangled in detail or nerves too big, it is difficult to get along with people happy. We introduced the concept of soft margin when the data for our training set had to be segmented with very complex curves.

In SVM without introducing soft margin, we want the data points in each training set to meet at least the following conditions, that is, the function distance from margin is greater than 0, that is, the distance hyperplane function distance is greater than 1

And considering that if the function distance of some outliers points is less than our expectation, the deviation is ξ, then these points satisfy the condition:

So, we put the previous optimization problem as follows:

Conversions to:

That is to say, on the one hand we need to optimize Ω, making margin=1/| | Ω| | The value is maximized, and on the other hand we choose ω to make the outliers of the deviation of the sum of the smallest, in between the two to seek a balance. C is the balance factor used to adjust the weights between the two-part adjustment items. The optimized Lagrangian function is:

After seeking duality, using KKT condition:

With the return to the original L function, the coefficients of theξ become C-α-r=0 and thus are eliminated, so the dual problem becomes:

As you can see, the form is almost the same as the original problem, and the knowledge adds an upper limit of C to the α condition.

Support Vector Machine (3): The beauty of Soft Margin balance

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.