Libsvm User Guide in MATLAB environment

 

I. Installation

Http://www.csie.ntu.edu.tw /~ Cjlin/libsvm/Matlab /. The following source code contains the MATLAB interface. After the download, you can put it on any disk and decompress it. It is best to create a folder dedicated to the MATLAB program. Open MATLAB, find the set path option on the MATLAB Panel (different versions of r2010b are different under the File menu), and add the downloaded libsvm directory to the MATLAB search path. Then, locate the current path to the libsvm/MATLAB directory (in the current floder (you can set it above the middle Of the MATLAB interface )). If your computer is 64-bit, you do not need to compile and generate the Mex file because the file has been compiled in advance. If it is 32-bit, you need to click make. M file to run the compilation to generate the Mex file. This step may fail. You can enter Mex-setup to check and select the C ++ compiler. If the compiler cannot be found, download and install Visual Studio. After compilation, libsvm can be used in any MATLAB program.

Ii. Use

Before using it, you need to understand libsvm, and the best information is the libsvm readme file. The readme file contains the introduction, installation, model parameters returned by SVM, and examples. Libsvm mainly uses two functions: svmtrain and svmpredict. If you forget the specific parameters, you can enter the function name at the MATLAB command prompt.

Usage: 1. Model = svmtrain (training_label_vector, training_instance_matrix [, 'libsvm _ options']);

(1)Training_label_vector, training_instance_matrix:

Training_label_vector and training_instance_matrix are the tags and features to be trained. Training_label_vector is generally stored in the form of column vectors. Each element is the tag that each row represents an instance (generally a feature. Training_instance_matrix is similar to a row that represents an instance.

 

The variables above indicate that there are 930 instances for training, and each instance1116 dimensions, the following is the instance tag.

(2) Libsvm_options:

How to choose? Libsvm_options:-T and-V during cross-validation.

-S SVM type: Specifies the SVM setting type. Generally, no value is required by default.

0 -- C-SVC (multiclass classification) 1 -- V-SVC (multiclass classification) 2-class SVM 3 -- e-SVR 4 -- V-SVR

-T-core function type: Core Function setting type (default value: 2)

0-linear: u'v 1-polynomial :( R * U' V + coef0) ^ degree 2-RBF function: exp (-Gamma | U-v | ^ 2)

3-sigmoid: Tanh (R * u'v + coef0)

-D degree: degree setting in the kernel function (for polynomial kernel function) (default value: 3)

-G r (GAMA): Gamma function settings in the kernel function (for polynomial/RBF/sigmoid core function) (1/K by default)

-R coef0: coef0 setting in the kernel function (for polynomial/sigmoid kernel function) (0 by default)

-C Cost: set parameters for C-SVC, e-SVR, and V-SVR (loss function) (default 1)

-N nu: set V-SVC, a class of SVM and V-SVR parameters (0.5 by default)

-P: Set the loss function P value in e-SVR (0.1 by default)

-M cachesize: Set the cache memory size, in MB (40 by default)

-E eps: Set the allowable termination criteria (0.001 by default)

-H shrinking: whether to use heuristic, 0 or 1 (default value: 1)

-Wi weight: Set parameter C of the nth type to weight * C (C in the C-SVC) (default 1)

-V n: N-fold interaction test mode. N indicates the number of fold, which must be greater than or equal to 2.

(3) Returned Model:

For example,-parameters: parameter.

-Nr_class: Number of classes.

-Totalsv: Total number of SVM.

-ROV:-The decision function wx + B.

-Label: the label of each class.

-PROBA: pair probability information. If B is 0, it is null.

-Probb: pair probability information. If B is 0, it is null.

-NSv: Support Vector for each class

-Sv_coef: coefficient of the decision function

-SVS: Support Vector.

If '-V' is specified, cross-validation is implemented and the returned result is the correct rate of cross-validation.

Usage:

2. [predicted_label, accuracy, decision_values/prob_estimates] = svmpredict (testing_label_vector, testing_instance_matrix, model [, 'libsvm _ options']);

(1) Esting_label_vector, testing_instance_matrix:

Similar to svmtrain

(2) Libsvm_options:

-B probability estimation: The default value is 0, 1, indicating estimation. (Explicit expression is required during use)

(3) Predicted_label, accuracy, decision_values/prob_estimates:

Predicted_label: SVM prediction output vector. Similar to the input label

Accuracy: The vector includes the accuracy rate, mean square error, and square correlation coefficient.

Decision_values: each row is the result of prediction K (k-1)/2 class II SVMs

Prob_estimates:Each row includes KValues represent the probability of testing data in each type.

Iii. Core functions

The purpose of a kernel function is to map feature vectors to a high-dimensional space. SVM searches for the largest spacing classification hyperplane in a high-dimensional space.

There are four types of core functions in total. We need to know under what circumstances which core function is used.

Generally, the first choice is the RBF kernel, which can map samples to a high-dimensional space in a non-linear manner, so it can handle situations where class labels and features are not linearly related. Linear kernel is only a special case of RBF.

However, in some cases, the RBF kernel is not applicable. For example, when the number of features is very large, only linear kernels are used.

Iv. Parameter Adjustment-cross verification and grid search

The RBF kernel has two parameters to be determined. We don't know what is best suited to given problems. The most common method is cross-validation, which divides the training data into two parts. The other part is unknown. For example, in the V-fold Cross verification, the training data is first divided into V subsets of the same size. Then training the classifier with the training data of the V-1 subset and testing the classifier with the remaining subset. Cross-validation can solve the problem of over-fitting. How can we use cross-validation to select the best one? It is generally used to search by grid, and the coordinate system has been divided into a series of grids. Then, use the vertices on the grid (for each node) for cross verification, and take the parameters with the highest accuracy rate for the last cross verification as the final parameter.

PS: when to use RBF and linear kernel

1. When the number of samples is much smaller than the number of features

For example, when there are 20 or 30 training and test data and thousands of feature dimensions, using a linear kernel is the best, and data does not need to be mapped.

2. The number of samples and the number of features are very large.

You can use another toolbox, liblinear, or linear kernel.

3. When the number of samples is much greater than the number of features

It is best to use a non-linear kernel.

References: http://www.csie.ntu.edu.tw /~ Cjlin/papers/GUIDE/guide.pdf