Understanding of support vector machine algorithm

Support Vector Machine (SVM) is a two classification model. Its basic model is a linear classifier that defines the largest interval in the feature space, which distinguishes him from the perceptual machine, and the support vector machine also includes the kernel technique, which makes it a substantially nonlinear classifier. The learning strategy of support vector machine is to maximize the interval. Can be formatted as a problem to solve convex two-time planning.

When the training data is linearly separable, by maximizing the hard interval, a linear classifier is learned, i.e. a linear separable support vector machine

When the training data is approximated linearly, by maximizing the soft interval, a linear classifier is also learned, that is, the linear support vector machine

When the training data is linearly non-tick, the nonlinear support vector machine is learned by using the kernel technique and maximizing the soft interval.

Nuclear method, nuclear technique is the use of kernel functions equal to implicitly learning linear support vector machines in high dimensional feature space.

First question: How to find the maximum hard interval

That is, from the support point to the ultra-plane distance is the largest, converted into a constrained optimization problem,

This constraint problem is also a convex two-time planning problem. Convert Chengrangerange problem, then to even function, in can use KKT condition, find out W,b

Second question: How to find the maximum soft interval

Linear irreducible means that some sample points (x, y) cannot satisfy the constraint of a function interval greater than or equal to 1, in order to solve this problem

Relaxation variables need to be added to the constraints, because some support vector machine points are divided incorrectly. So the constraint is the >=1-relaxation variable (relaxation variable "0)

Then continue to pass the penalty function method, the objective function adds a coefficient, as long as the penalty of the wrong classification increases, the penalty of the C-value hour for the mis-classification decreases.

The third problem kernel function

Linear classification Support Vector machine is a very effective method for solving linear classification problem. But sometimes the classification problem is non-linear, then the nonlinear support vector machine can be used. The kernel function is to map the input space to the feature space, through the inner product between the eigenvectors, by using the kernel function to learn the non-support vector machine

Commonly used kernel functions are: polynomial kernel function, Gaussian kernel function, string kernel function.

As mentioned earlier, the convex two-time programming problem has the global optimal solution, but when the training sample capacity is very large, these algorithms are often very inefficient. So the descendant proposes the sequence minimum optimization (SMO) algorithm.

This article from "Jane Answers Life" blog, declined reprint!

Understanding of support vector machine algorithm