Introduction to SVM

SVM is proposed from the optimal classifier plane in linearly segmented situations. The so-called optimal classification requires that the classification line not only separates the two types without errors, but also has the largest classification interval between the two types. The former ensures the minimum empirical risk (0 ), we can see from the discussion below that the maximum classification interval is actually to minimize the confidence range in the promotion. When extended to a high-dimensional space, the optimal classification line becomes the optimal classification surface.

SVM uses the idea of classification interval for training. It relies on data preprocessing, that is, expressing the original pattern in a higher-dimensional space. By appropriate ing to a non-linear ing with sufficient high dimensions, two types of raw data can be separated by a hyperplane. As shown in:

 

The hollow points and solid points represent two different classes. H is the classification surface that separates the two classes without errors. H is also an optimal classification surface. The reason is as described above. When H is used as the classification surface, the classification interval is the largest and the error is the smallest. The distance between the two classes is the classification interval between the two classes. SVM maps data from the original space to a high-dimensional space to find an optimal classification surface, which maximizes the classification interval margin. The training samples that define the optimal classification over the plane, that is, the hollow points and solid points in the middle, are the support vectors mentioned in support vector machine theory. Obviously, the so-called SVM are actually the most difficult vectors to be classified. However, from another perspective, they are also the most valuable pattern for solving classification tasks.

The basic idea of SVM can be summarized as follows: first, the input space is transformed to a high-dimensional space through nonlinear transformation, and then the Optimal Linear Classification surface is obtained in this new space, this kind of nonlinear transformation is implemented by defining appropriate inner product functions. The classification function obtained by SVM is similar to a neural network. Its output is a linear combination of several intermediate layer nodes, and each intermediate layer node corresponds to the Inner Product of the input sample and a support vector, therefore, it is also called Support Vector Network. As shown in:

 

 

Because the final discriminant function only contains the Inner Product and sum of the Support Vector, the computational complexity of the discriminant classification depends on the number of support vectors.

It is not difficult to find that SVM, as a classic representative of statistical learning theory, uses a completely different idea from the traditional method, that is to say, it is not the traditional method that first tries to reduce the dimension of the original input space (that is, feature selection and feature transformation), but tries to increase the input space to the dimension, in order to make the problem in high-dimensional space linear or near linear. Because the increasing dimension knowledge changes the inner product operation, the complexity of the algorithm does not increase with the increase of the dimension, and the promotion capability in the high-dimensional space is not affected by the dimension.

In addition, it should be noted that SVM uses different inner product functions, which will lead to different SVM algorithms

Currently, the inner product functions have the following three types:

(1) internal product functions in the form of polynomials;

(2) Inner Product functions in the form of core functions;

(3) Use the S-shaped function as the inner product function;