Python vs machine learning-support vector machines

The basic model of support vector machines (SVM MACHINE,SVM) is a linear classifier that defines the largest interval in a feature space. It is a two-class classification model, and the support vector machine can be used for nonlinear classification after the kernel technique is adopted.

1) linear can be divided support vector machine (also called hard interval Support vector machine): When the training data linear can be, through the hard interval maximization, learned a linear can be divided into support vector machine

2) linear support vector machine (also known as soft interval support vector machine): When the training data is approximately linear, it is maximized by the soft interval to learn a linear support vector machine

3) Nonlinear support vector machine: When the training data is not available, a nonlinear support vector machine is learned by using the kernel technique and maximizing the soft interval.

1, linear can be divided into support vector machine

Input: Linear training Data Set T

Output: Separation of super-planar and categorical decision-making functions for maximum geometric interval

Algorithm steps:

1) construct and solve the constrained optimization problem, obtain the optimal solution w*,b*

2) The separation of the super-plane, and the classification decision-making function

If the training data set T is linearly separable, the maximum interval separates the presence of the hyper plane and the unique

The following is a dual algorithm for the linear scalable support vector machine learning algorithm:

Input: Linear training Data Set T

Output: Separation of super-planar and categorical decision-making functions for maximum set interval

Algorithm steps:

1) construct and solve the constrained optimization problem and obtain the optimal solution α*

2) Calculate the w*, and select a positive component of the α* αj*>0, calculate b*

3) The maximum set-interval separation of super-plane and categorical decision-making function is obtained.

2. Linear Support Vector machine

Linear support vector machines are no longer suitable for linearly non-divided training data, but can be extended to linear non-fractal problems

Dual algorithm of linear support vector machine learning algorithm:

Input: Training data set T, penalty parameter c>0

Output: Soft interval maximizes separation of hyper-plane and categorical decision functions

Algorithm steps:

1) Solve the constrained optimization problem and obtain the optimal solution α*

2) Calculation w*,b*

3) This results in the maximum separation of the soft interval and the classification decision function.

Experiment Code:

1 ImportMatplotlib.pyplot as Plt2 ImportNumPy as NP3  fromSklearnImportDATASETS,LINEAR_MODEL,CROSS_VALIDATION,SVM4 5 defload_data_regression ():6diabetes=datasets.load_diabetes ()7     returnCross_validation.train_test_split (diabetes.data,diabetes.target,test_size=0.25,random_state=0)8 9 defload_data_classfication ():Teniris=Datasets.load_iris () Onex_train=Iris.data Ay_train=Iris.target -     returnCross_validation.train_test_split (x_train,y_train,test_size=0.25,random_state=0,stratify=Y_train) -  the defTEST_LINEARSVC (*data): -x_train,x_test,y_train,y_test=Data -cls=SVM. Linearsvc () - Cls.fit (X_train,y_train) +     Print("coefficients:%s,intercept%s"%(cls.coef_,cls.intercept_)) -     Print("score:%.2f"%Cls.score (x_test,y_test)) +  Ax_train,x_test,y_train,y_test=load_data_classfication () atTest_linearsvc (X_train,x_test,y_train,y_test)

View Code

Experimental results:

The accuracy of the predictions on the test set reached 0.97, or very high

3. Nonlinear Support vector machine

The kernel function maps any two vector x,z in the input space into the inner product between the corresponding vectors in the feature space. In the case of a given kernel function K (x,z), a support vector machine for nonlinear classification problem can be solved by solving the problem of linear classification.

In practical application, it is often relied on experience to select kernel function directly, and then verify that the kernel function is indeed a valid kernel function. The usual kernel functions are as follows:

1) polynomial kernel function 2) Gaussian kernel function 3) sigmoid kernel function

Input: Training data set T, penalty parameter C

Output: categorical decision function

Algorithm steps:

1) Select the appropriate kernel function k to solve the constrained optimization problem and obtain the optimal solution α*

2) Calculate w* and b*

3) Constructing categorical decision functions

4. Support vector regression (supported vectors regression,svr)

5. The advantages and disadvantages of SVM

SVM is a nonlinear method in nature, it is easy to grasp the nonlinear relationship between the data and features when the sample size is very small, so it can solve the nonlinear problem, avoid the neural network structure selection and local minimum problem, improve generalization performance and solve the high dimension problem.

SVM is sensitive to missing data, and there is no universal solution for nonlinear problems, so we must choose kernel function carefully to deal with it, and the computational complexity is high.

　　　　

Python vs machine learning-support vector machines