[Machine Learning SVM] linear SVM or non-linear SVM? [Switch]

SVM is widely used in classification, regression, density estimation, clustering, etc. But I think the most successful is classification.

When used for classification problems, there are not many parameters available for SVM. The penalty parameter C, kernel function, and parameter selection are. For an application, is linear kernel, polynomial kernel, or Gaussian Kernel selected? There are still some rules.

In practice, most cases feature dimensions are very high. For example, in OCR, Chinese character recognition is used to extract eight-direction gradient histogram features. Normalized characters are classified into 8*8 grids. Each grid computes an 8-direction histogram, the feature dimension is 8x8x8 = 512. In such a high-dimensional space, it is easy to use linear SVM to separate the two character classes. Of course, they can also be separated with other cores. So why choose a linear kernel, because the linear kernel has two major advantages:

1. Simple Prediction function f (x) = W' * x + B, fast classification. For the problem of many categories, the classification speed must be taken into account. The W of the linear classifier can be calculated in advance, while the nonlinear classifier can support a large number of vectors in a high-dimensional space, classification speed is much lower than linear classifier.

2. The promotion of linear SVM is guaranteed, while non-linearity such as Gaussian Kernel may be over-learned. Another example is face-based gender recognition, that is, determining whether a person is male or female based on a given face image. We have extracted 3700-dimensional features, and linear SVM can achieve 96% recognition accuracy in the test set. Therefore, linear SVM is the most widely used and has the greatest practical value.

3. In your application, if the feature dimension is extremely low and the number of samples far exceeds the feature dimension, it is reasonable to use a non-linear kernel such as Gaussian Kernel. If the two types overlap a lot, there are a lot of support vectors for the non-linear SVM. It is a better solution to select a sparse nonlinear SVM, and the support vector is faster in less classification, for example:

650) This. width = 650; "src =" http://fmn.rrimg.com/fmn059/20120507/1045/ B _large_Dc4N_26e600002f1b1263.jpg "alt =" B _large_dc4n_26e600002f1b1263.jpg "/>

Now I have a basic understanding of linear SVM or non-linear SVM? The above are some of my personal understandings, which may lead to deviations. You are welcome to correct or supplement them. I will continue to write some SVM posts later, hoping to be helpful to those who are interested.

From: http://xiaozu.renren.com/xiaozu/121443/356866219

This article is from the squirrel blog, please be sure to keep this source http://apinetree.blog.51cto.com/714152/1560240

[Machine Learning SVM] linear SVM or non-linear SVM? [Switch]