Support Vector Machine (SVM) Algorithm

Support Vector Machine (SVM) is a classification algorithm that seeks to minimize structural risks to improve the generalization ability of learning machines and minimize empirical risks and confidence ranges, in this way, a good statistical rule can be obtained when the sample size is small. Generally speaking, it is a second-class classification model. Its basic model is defined as the linear classifier with the largest interval in the feature space. That is, the learning strategy of SVM is to maximize the interval, finally, it can be converted into solving a convex quadratic programming problem.

Principle:

1. Find a classification hyperplane in the n-dimensional space and classify the points in the space. This is an example of Linear classification.

2. In general, the distance between a point and the superplane can be expressed as the confidence or accuracy of classification prediction. SVM is used to maximize the interval value. The dot on the dotted line is called the Support Vector supprot verctor.

3. In practice, we often encounter linear samples that cannot be divided. In this case, we usually map the sample features to a high-dimensional space (for example );

3. Linear ing to a high-dimensional space may lead to a terrible dimension height (19-dimensional or even infinite-dimensional examples), resulting in complicated computing. The value of a core function is that, although it also refers to the conversion of features from low-dimensional to high-dimensional, the core function is always computed on low-dimensional aspects in advance, as mentioned above, the classification effect is represented in high dimensions, which avoids complex computing directly in high-dimensional spaces.

4. Use relaxation variables to process data Noise

Advantages of SVM:

1. The SVM learning problem can be expressed as a convex optimization problem. Therefore, we can use known effective algorithms to find the global minimum value of the target function. Other classification methods (such as rule-based classifier and Artificial Neural Networks) Use a greedy learning-based strategy to search for hypothetical spaces. Generally, this method can only obtain local optimal solutions.

2. Assume that you are a farmer and have a group of sheep in custody, but to prevent the wolves from attacking the herd, you need to build a fence to enclose them. But where should the fence be built? You may need to create a classifier based on the positions of the herd and the Wolf group. By comparing these classifier types, we can see that SVM has completed a perfect solution.

This example illustrates the advantages of SVM in using a non-linear classifier, while the logic mode and decision tree mode both use the linear method.

 

Excerpted from http://blog.csdn.net/v_july_v/article/details/7624837.