Explanation of model parameters after LIBSVM training

The main point is to explain the use of the Libsvm-mat Toolbox to establish a classification (regression model), the model parameters obtained in the meaning are god horse? And if the model is given an expression, the classification problem is the main example.

The test data uses Libsvm-mat's own Heart_scale.mat data (an attribute matrix of 270*13, a total of 270 samples, each with 13 attributes), to facilitate testing and learning.

First, the last short test code:

2. Percent Modeldecryption
4. % by Faruto @ Faruto ' s studio~
6. % Http://blog.sina.com.cn/faruto
8. % Email:[email protected]
10. % http://www.matlabsky.com
12. % http://www.mfun.la
14. % http://video.ourmatlab.com
16. % last modified by 2011.01.06
18. Percent a litte clean work
20. Tic
22. Close all;
24. Clear
26. CLC
28. Format Compact;
30. %%
32. % Load Data First
34. Load Heart_scale;
36. data = Heart_scale_inst;
38. label = Heart_scale_label;
40. % Build a classification model
42. Model = Svmtrain (Label,data, '-s 0-t 2-c 1.2-g 2.8 ');
44. Model
46. % use the established model to see its classification effect on the training set
48. [Predictlabel,accuracy] = Svmpredict (Label,data,model);
50. Accuracy
52. %%
54. ToC

Operation Result:

1. Model =
2. Parameters: [5x1 Double]
3. Nr_class:2
4. totalsv:259
5. rho:0.0514
6. Label: [2x1 double]
7. ProbA: []
8. Probb: []
9. NSV: [2x1 double]
10. SV_COEF: [259x1 Double]
11. SVs: [259x13 Double]
12. accuracy = 99.6296% (269/270) (classification)
13. accuracy =
14. 99.6296
15. 0.0148
16. 0.9851
17. Elapsed time is 0.040155 seconds.

Here for the sake of simplicity does not have the test data for training set and test set division, here is only for simple and clear, the classification results are estimated can not tube, parameter optimization also do not tube, another post explained.

Let's take a look at the model in the structure of the various parameters within the meaning of the divine Horse, themodel is as follows:

Model =
Parameters: [5x1 Double]
Nr_class:2
totalsv:259
rho:0.0514
Label: [2x1 double]
ProbA: []
Probb: []
NSV: [2x1 double]
SV_COEF: [259x1 Double]
SVs: [259x13 Double]

Model. Parameters

Let's look at the model first. What are the parameters inside the pack:

1. >> model. Parameters
2. Ans =
3. 0
4. 2.0000
5. 3.0000
6. 2.8000
7. 0

Important points of knowledge:

Model. Parameters the meaning of the parameter from top to bottom:
-S SVM Type:SVM Setting type(Default0)
-T kernel function type: kernel function set type(Default2)
-D degree: in the kernel functiondegree settings (for polynomial kernel functions 3)
-G R ( GAMA): gamma function in kernel function set (for polynomial /rbf/sigmoid kernel function -R coef0: coef0 settings in kernel functions /sigmoid kernel function 0)
That is, in this case through model. Parameters we can learn  –s  parameter is 0;-t  parameter is  < Span lang= "en-US" >2;-d  parameter is 3;-g  parameter is 2.8 (this is also our own input), -r  parameter is 0.

A little description of the >LIBSVM parameter of

The parameter settings in the LIBSVM can be arbitrarily combined according to the type of SVM and the parameters supported by the kernel function, and the program will not accept the parameter if it does not have an effect on the function or SVM type, and the parameter will take the default value if the expected parameter is set incorrectly.
Model. Label Model.nr_class

1. >> model. Label
2. Ans =
3. 1
4. -1
5. >> Model.nr_class
6. Ans =
7. 2

Important points of knowledge:

Model. Label indicates what the label of the category in the DataSet has, here is 1,-1;
Model.nr_class indicates how many categories are in the dataset, and here is the two classification.

MODEL.TOTALSV MODEL.NSV

1. >> MODEL.TOTALSV
2. Ans =
3. 259
4. >> MODEL.NSV
5. Ans =
6. 118
7. 141

Important points of knowledge:

MODEL.TOTALSV represents the total number of support vectors, there are 259 support vectors;
MODEL.NSV represents the number of support vectors for each class of samples, which indicates that the support vector for the sample labeled 1 is 118, and the support vector for the sample labeled-1 is 141.
Note: Here MODEL.NSV is represented by the order and model. Label corresponding to the.

Model. ProbA model. Probb

About these two parameters are not introduced here, using the -B parameter can be used, for probability estimation.

-B probability_estimates:whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)

MODEL.SV_COEF model. SVs Model.rho

1. SV_COEF: [259x1 Double]
2. SVs: [259x13 Double]
3. Model.rho = 0.0514

Important points of knowledge:

Model.sv_coef is a 259*1 matrix that is loaded with the coefficients of 259 support vectors in the decision function;
Model. SVS is a sparse matrix of 259*13 that is loaded with 259 support vectors.
Model.rho is the inverse of a constant term in a decision function (-b)

First of all, let's look at the expression of the final categorical decision function obtained by the –s 0 parameter (c-svc model).
If there are any areas where the C-SVC model is not understood, please see this PDF file:
Libsvm_library.pdf
Attachment:
The final decision function is:
Since we are using the RBF kernel function (the previous parameter setting –t 2), the decision function here is:
where | | x-y | | is two norm distance ;
It's inside.

B is -model.rho (a scalar number);
b =-model.rho;
n means the number of support vectors, n = MODEL.TOTALSV (a scalar number);
For each of the I:
WI =model.sv_coef (i); Coefficient of support vector (a scalar number)
XI = model. SVs (i,:) Support vectors (line vectors for 1*13)
X is a sample of the label to be predicted (1*13 line vector)
Gamma is the- g parameter
Okay, so let's create the above decision function by using the model information provided below:

2. Percent Decisionfunction
4. function Plabel = decisionfunction (X,model)
8. Gamma = model. Parameters (4);
10. RBF = @ (u,v) (exp (-gamma.*sum ((u-v). ^2));
14. Len = Length (MODEL.SV_COEF);
16. y = 0;
20. For i = 1:len
22. U = model. SVs (i,:);
24. y = y + model.sv_coef (i) *RBF (u,x);
26. End
28. b =-model.rho;
30. y = y + b;
34. If y >= 0
36. Plabel = 1;
38. Else
40. Plabel =-1;
42. End

With this decision function, we can predict the label of the corresponding sample ourselves:

1. %%
2. plable = zeros (270,1);
3. For i = 1:270
4. x = Data (i,:);
5. Plabel (i,1) = Decisionfunction (X,model);
6. End
8. Percent percent verifies that the label that you predict by the decision function is the same as the label given by Svmpredict
9. Flag = SUM (Plabel = = Predictlabel)
10. over = 1;

The

can finally see the  flag = 270 , that is, the decision function established by itself is correct and can be obtained and svmpredict the bottom is generally the same way to achieve.
Finally, let's take a look at the meaning of the return parameter svmpredict
 

1. %%
2. % Load Data First
3. Load Heart_scale;
4. data = Heart_scale_inst;
5. label = Heart_scale_label;
6. % Build a classification model
7. Model = Svmtrain (Label,data, '-s 0-t 2-c 1.2-g 2.8 ');
8. Model
9. % use the established model to see its classification effect on the training set
10. [Predictlabel,accuracy] = Svmpredict (Label,data,model);
11. Accuracy

Run can see

1. Model =
2. Parameters: [5x1 Double]
3. Nr_class:2
4. totalsv:259
5. rho:0.0514
6. Label: [2x1 double]
7. ProbA: []
8. Probb: []
9. NSV: [2x1 double]
10. SV_COEF: [259x1 Double]
11. SVs: [259x13 Double]
12. accuracy = 99.6296% (269/270) (classification)
13. accuracy =
14. 99.6296
15. 0.0148
16. 0.9851

Here, let's talk about the meaning of the three parameters of the return parameter accuracy.
Important points of knowledge:

return parametersThe meanings of accuracy from top to bottom are:
Classification quasi-rate (parameter indicator used in classification problem)
Average squared error (MSE (mean squared error))[parameter indicators used in regression problems]]
Squared correlation coefficient (R2 (squared correlation coefficient))[regression problem ]
where mse  and The formulas for R2 are: Illustrations:
in the following case, this is about the meaning of the corresponding parameters in model, And even if it is clear that the expression of the decision-making function or the method of calculation is given.
Maybe some classmates will ask, how to get the classification decision-making function of the alpha coefficient "This will certainly be asked", or again grind to talk about it:
above wi is actually alpha and Support vector category tags (1 or Span lang= "en-us" >-1), the original decision function expression is as follows:
Illustration:
Yi is a category tag that supports vectors (1 or -1), libsvm in yi and The product of span lang= "en-US" >alpha is put together with model.sv_coef (w) to mount.
All speaking of this, should be able to understand?

More nonsense: In the study of SVM, I found a lot of friends are not active thinking and try, always ask, so very bad, so very passive, above these also no one taught me, are I think it out, and then programming verification, if there is unreasonable place to continue to think, Anyway, the truth and books are explained, can always penetrate the AH. o (∩_∩) O?

Transferred from: http://blog.sina.com.cn/s/blog_6646924501018fqc.html

Explanation of model parameters after LIBSVM training (turn)