SVM Support Vector Machine (1) theory knowledge of the Python implementation of machine learning algorithms

1. Background

Highly recommended reading (http://www.cnblogs.com/jerrylead/archive/2011/03/13/1982639.html)

Support Vector machine SVM (support vector machines). SVM is a binary classifier, which is a popular classification algorithm in recent years.

This article, first of all to introduce some basic knowledge concepts, in the next chapter will be a simple code implementation of SVM.

2. Basic Concepts

(1) linear can be divided

First of all introduce what is called linear, reference to a picture in the previous section. The linear division is the fact that you can distinguish between two different points in a single line. Thus we can get the linear irreducible is two kinds of points mixed together cannot distinguish. But the point where the line is not distinguishable can be distinguished by mathematical methods. For example, a four-dimensional dataset we can use a three-dimensional object to separate it, this object is called the hyperplane. The super plane of the image below is the Blue line.

(2) Support vector

Support Vector, now that we know the concept of hyperplane, the support vector is actually the nearest vector from the hyperplane. The above picture, for example, is the nearest point from the Blue Line. The method is to determine the distance from the point to the line. Once we have found these support vectors, we can zoom in on these vectors and consider only those objects, using the idea of minimal sequence optimization.

(3) Lagrange multiplier method

For the support vector method, we need some constraint conditions. For example, the distance we have to the hyperplane is D, and we need to take the d>1 point as a constraint. Because if this constraint does not occur, the calculation will be error.

The formula is the set of points that we go to the smallest point to the hyperplane, and satisfies

。 In the existence of constraints in the case of the problem of extremum, we use the Lagrange multiplier method (see Baidu Encyclopedia).

(4) Variant

Reference to Lagrange formula F (x1,x2,... λ) =f (x1,x2,...) -λg (x1,x2 ...). We've transformed the above equation into

This column more highlights: http://www.bianceng.cnhttp://www.bianceng.cn/Programming/sjjg/

The constraint becomes

The upper-type parameter C makes the relaxation variable because we see some of the red dots in the graph being divided into the green point range, in order to consider this problem, introduce a variable to control. The main task of SVM is to compute parameter C.

Author: csdn Blog Rae Garvin