First-Entry SVM learning report

Study Report

1 Work this week

Preliminary understanding of SVM, familiar with the process of the algorithm, build models, and write program implementation.

Understand the meaning of each line of code (for example, wine experiment and Shanghai Composite Index)

Understanding the method of cross selection parameters, and the meaning

Solve the problem of software running and compatibility, lay a good foundation for the experiment

2 Experimental Summary

(1) Algorithm Introduction

(1) Introduction of SVM

Support Vector machine: (SVM) similar to multilayer perceptron network, radial basis function network for pattern classification and linear regression. The main idea of support vector machine is to establish a super class plane as a decision-going surface, so that the edge between the positive case and the inverse case can be maximized to make the distinction.

1. Multilayer perceptual Networks: A Feedforward artificial neural network model that maps multiple data sets entered into a single output dataset.

2. Radial basis function: It is a real value function which depends only on the distance from the origin point.

3. Robustness: Refers to the control system in a certain (structure, size) of the parameter perturbation, to maintain some other performance characteristics.

4.VC Dimension: The so-called VC dimension is a measure of the function class, you can simply understand the complexity of the problem, the higher the VC dimension, the more complex a problem.

5. Kernel function: Polynomial kernel function k (x,xi) = (x▪xi+1) ^d, d=1,2,..., N; According to the theory of pattern recognition, the linear irreducible model of low dimensional space can be linearly divided by nonlinear mapping to high dimensional feature space, However, if this technique is used to classify or return in high dimensional space, there are some problems, such as the form and parameter of nonlinear mapping function and the dimension of feature space, and the biggest obstacle is the "dimensionality disaster" in high-dimensional feature space operation.

Type:: Linear kernel function, polynomial kernel function, radial basis kernel function, sigmoid kernel function and composite kernel function.

(2) Wine classification, in order to achieve satisfactory results, need to adjust the relevant parameters (c[penalty parameters],g[function parameters)), at this time, we will use cross-validation, so that both to avoid the lack of learning, but also to avoid learning.

1.hold-out method: Randomly divided into two groups of training sets, a set of validation set. Using training set training classifier, validating set to validate the model, the accuracy rate is related to the grouping, so it is not used generally.

2.KCV: The original data is divided into K group, each subset of data is validated separately, the remaining K-1 group is the training set. K is generally greater than 2, only a small amount of data to use 2.

3.N-CV: Each sample as a separate validation set, the rest of the N-1 as a training set, so you will get N models, the experimental results have a high degree of persuasion, but the computational volume is very large.

Genetic algorithm is a computational model of the natural selection and genetic mechanism of Darwin's evolutionary evolution process, and is an optimal solution by simulating natural evolution and process search.

Genetic algorithms simulate the replication, crossover, mutation and other phenomena, from any group, through random crossover, mutation, duplication and other operations, to create a group of more suitable for the environment of individuals, so that the group evolved into a better area of search space, so that generation of evolution, a more adaptable to the environment of the individual, namely the optimal solution

(3) The algorithm adopted is the first two methods, firstly, using the original data of the fuzzy information granulation, and finally adding the regression prediction. The regression analysis was performed by the 4,579 data provided in the book. Firstly, the dependent variables and fruit variables are selected, the data is normalized and the optimal parameter c/g is selected by using K-CV, then the SVM is trained and the fitting prediction is finally carried out.

(2) Problem description

(1) using SVM to classify Italian wines and data using UCI's wine data. A total of 178 data, of which 50% is a test set, 50% is a training set. The training set is used to train SVM to get the classification model and to predict the test set by using the obtained model. Then the same kernel function is used to test the accuracy rate with different normalization methods, and the same normalization is used to test the accuracy of different kernel functions.

(2) in the classification of wine by SVM, using K-CV to select the best parameters C, g

(3) using SVM to predict regression, based on the data from 1990 to 2009 provided by the Great Wisdom Software, select the highest value, lowest value, closing index, trading volume of the opening index from 1 to 4578, to the turnover as the independent variable, select the second to 4579 in the index as the dependent variable, Then the data is normalized and the last K-CV is selected to select the best parameter, then the SVM is trained and the regression is predicted.

(3) Program flow

First, using SVM to classify wine

Second, using K-CV to optimize the parameters of the previous experiment

Third, SVM regression prediction

(4) Program realization

1. Key codes for the classification of wines

Close all;

Clear

CLC

Format Compact;

% percent Data extraction

Load Chapter12_wine.mat;

% selected training sets and test sets

% will be the first class of 1-30, the second category of 60-95, the third class of 131-153 as a training set

Train_wine = [Wine (1:30,:); wine (60:95,:); wine (131:153,:));

% of the corresponding training set label also to be separated out

Train_wine_labels = [Wine_labels (1:30); Wine_labels (60:95); Wine_labels (131:153)];

% will be the first class of 31-59, the second class of 96-130, the third class of 154-178 as a test set

Test_wine = [Wine (31:59,:); wine (96:130,:); wine (154:178,:));

% the label of the corresponding test set should also be separated.

Test_wine_labels = [Wine_labels (31:59); Wine_labels (96:130); Wine_labels (154:178)];

% percent data preprocessing

% data preprocessing, normalization of training set and test set to [0,1] interval

[Mtrain,ntrain] = size (train_wine);

[Mtest,ntest] = size (test_wine);

DataSet = [Train_wine;test_wine];

% Mapminmax is a normalized function with Matlab

[Dataset_scale,ps] = Mapminmax (DataSet ', 0, 1);

Dataset_scale = Dataset_scale ';

Train_wine = Dataset_scale (1:mtrain,:);

Test_wine = Dataset_scale ((mtrain+1):(mtrain+mtest),:);

%-percent SVM Network training

Model = Svmtrain (train_wine_labels, Train_wine, '-C 2-g 1-t 2 ');

Prediction of%-percent SVM Network

[Predict_label, Accuracy,dec_values] = Svmpredict (Test_wine_labels, test_wine, model);

% percent Result analysis

Figure

Hold on;

Plot (test_wine_labels, ' o ');

Plot (Predict_label, ' r* ');

Xlabel (' Test set Sample ', ' FontSize ', 12);

Ylabel (' category label ', ' FontSize ', 12);

Legend (' actual test set classification ', ' Predictive Test set classification ');

Title (' The actual classification and prediction of the test set ', ' FontSize ', 12);

Grid on;

After%%%, the different normalization was changed to experiment again.

[Dataset_scale,ps] = Mapminmax (DataSet ', -1,1);

%%% change the different kernel function to experiment

Model = Svmtrain (train_wine_labels, Train_wine, '-C 2-g 1-t 0 ');

Model = Svmtrain (train_wine_labels, Train_wine, '-C 2-g 1-t 1 ');

Model = Svmtrain (train_wine_labels, Train_wine, '-C 2-g 1-t 2 ');

Model = Svmtrain (train_wine_labels, Train_wine, '-C 2-g 1-t 3 ');

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5

Random selection of CG for experiment

%% Empty environment variable

function Chapter13_gridsearch

Close all;

Clear

CLC

Format Compact;

% percent Data extraction

Load Chapter13_wine.mat;

% selected training sets and test sets

Train_wine = [Wine (1:30,:); wine (60:95,:); wine (131:153,:));

Train_wine_labels = [Wine_labels (1:30); Wine_labels (60:95); Wine_labels (131:153)];

Test_wine = [Wine (31:59,:); wine (96:130,:); wine (154:178,:));

Test_wine_labels = [Wine_labels (31:59); Wine_labels (96:130); Wine_labels (154:178)];

% percent data preprocessing

[Mtrain,ntrain] = size (train_wine);

[Mtest,ntest] = size (test_wine);

DataSet = [Train_wine;test_wine];

[Dataset_scale,ps] = Mapminmax (DataSet ', 0, 1);

Dataset_scale = Dataset_scale ';

Train_wine = Dataset_scale (1:mtrain,:);

Test_wine = Dataset_scale ((mtrain+1):(mtrain+mtest),:);

Percent selection of optimal SVM parameters C&g

BESTC = 100*rand (1);

BESTG = 200*rand (1)-100;

Percent% using the best parameters for SVM network training

cmd = [' C ', Num2str (BESTC), ' G ', Num2str (BESTG)];

Model = Svmtrain (train_wine_labels,train_wine,cmd);

Prediction of%-percent SVM Network

[Predict_label,accuracy,dec_values] = Svmpredict (Test_wine_labels,test_wine,model);

% Print test set classification accuracy rate

Total = Length (test_wine_labels);

right = SUM (Predict_label = = test_wine_labels);

Disp (' Print test set classification accuracy rate ');

str = sprintf (' accuracy =%g%% (%d/%d) ', accuracy (1), right,total);

Disp (str);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% call the child function to select%%%%%%%%%%%%%%%%%%

% first make a rough choice: the range of c&g changes is 2^ ( -10), 2^ ( -9),..., 2^ (10)

[BESTACC,BESTC,BESTG] = Svmcgforclass (train_wine_labels,train_wine,-10,10,-10,10);

% in the rough selection of the results of the detailed selection: C is 2^ ( -2), 2^ ( -1.5),..., 2^ (4), g of the range is 2^ ( -4), 2^ ( -3.5),..., 2^ (4),

[BESTACC,BESTC,BESTG] = Svmcgforclass (train_wine_labels,train_wine,-2,4,-4,4,3,0.5,0.5,0.9);

%%%%%%%%%%%%%%5%%%%%%%%%%%%%%%%

The regression forecast of Shanghai stock index

function Chapter14

Tic

Close all;

Clear

CLC

Format Compact;

Load Chapter14_sh.mat;

% Extract Data

[M,n] = size (SH);

TS = sh (2:m,1);

TSX = SH (1:m-1,:);

% data preprocessing, normalization of raw data

ts = ts ';

TSX = TSX ';

[Ts,tsps] = Mapminmax (ts,1,2);

% ts are transpose to conform to the data format requirements of the LIBSVM Toolbox

ts = ts ';

% Normalization of TSX

[Tsx,tsxps] = Mapminmax (tsx,1,2);

% to TSX to conform to the LIBSVM Toolbox data format requirements

TSX = TSX ';

The best SVM parameter c&g for the regression prediction analysis of%-percent selection

% first make a rough choice:

[BESTMSE,BESTC,BESTG] = svmcgforregress (ts,tsx,-8,8,-8,8);

% make fine selections based on a rough selection of result graphs:

[BESTMSE,BESTC,BESTG] = svmcgforregress (ts,tsx,-4,4,-4,4,3,0.5,0.5,0.05);

% Print Fine selection results

Disp (' Print fine selection result ');

str = sprintf (' best Cross Validation MSE =%g best c =%g best g =%g ', BESTMSE,BESTC,BESTG);

Disp (str);

Percent% using regression prediction to analyze the best parameters for SVM network training

cmd = [' C ', Num2str (BESTC), ' G ', Num2str (BESTG), '-s 3-p 0.01 '];

Model = Svmtrain (ts,tsx,cmd);

Regression prediction of%-percent SVM Network

[Predict,mse,decal_value] = Svmpredict (Ts,tsx,model);

predict = Mapminmax (' reverse ', predict ', tsps);

predict = Predict ';

% Print regression results

str = sprintf (' mean square error MSE =%g correlation coefficient R =%g%% ', MSE (2), MSE (3) *100);

Disp (str);

% percent Result analysis

Figure

Hold on;

Plot (ts, '-o ');

Plot (predict, ' r-^ ');

Legend (' Raw data ', ' Regression prediction data ');

Hold off;

Title (' Comparison of original data and regression forecast data ', ' fontsize ', 12);

Xlabel (' Trading Days (1990.12.19-2009.08.19) ', ' fontsize ', 12);

Ylabel (' opening number ', ' fontsize ', 12);

Grid on;

Figure

Error = Predict-ts ';

Plot (Error, ' Rd ');

Title (' Error chart (predicted data-original data) ', ' fontsize ', 12);

Xlabel (' Trading Days (1990.12.19-2009.08.19) ', ' fontsize ', 12);

Ylabel (' Error amount ', ' fontsize ', 12);

Grid on;

Figure

Error = (Predict-ts ')./ts ';

Plot (Error, ' Rd ');

Title (' Relative error graph (predicted data-original data)/original data ', ' fontsize ', 12);

Xlabel (' Trading Days (1990.12.19-2009.08.19) ', ' fontsize ', 12);

Ylabel (' Relative error amount ', ' fontsize ', 12);

Grid on;

Snapnow;

ToC

(5) Result analysis

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(6) Unresolved issues

1 Genetic algorithms don't get it.

2 PSO looking for the best parameter didn't understand

3 Random selection programs run 3 times in a row will be an error

4 print-dtiff-r600 result; What does it mean?

3 Reference Documents

Matab Neural Network 30 case analysis. Spein

MATLAB Chinese Forum

SCDN Forum Articles

4 Working next week

Resolve Unknown issues

Learning SVM's information granular sequential regression prediction, understand the program, and make the results