"Machine Learning Algorithm-python implementation" SVM Support vector Machine (1)-Introduction to theoretical knowledge

(Reproduced Please specify Source:http://blog.csdn.net/buptgshengod)

1. BackgroundHighly Recommended reading (http://www.cnblogs.com/jerrylead/archive/2011/03/13/1982639.html)SVM (Support vector machines).

SVM is a binary classifier, which is a popular classification algorithm in recent years.

In this paper, we first introduce some of the main concepts of knowledge, the next chapter will be a simple code implementation of SVM.

2. Basic Concepts
(1) linear can be divided first to introduce what is called linear can be divided, referring to a picture of the previous section.

Linear is actually able to distinguish two different points in a straight line.

Thus we can get the linear irreducible is the two kinds of points mixed together cannot differentiate.

But the linear irreducible points can in fact be distinguished by mathematical methods.

For example, a four-dimensional data set, we can separate it with a three-dimensional object called the Super plane .

The super plane is the blue line.

(2) Support Vector support vectors, we now know the concept of hyper-plane. Support vectors are in fact vectors that are far from the super plane in the near future.

For example, those points near the Blue line. The method is to determine the distance from the point to the line.

Once we have found these support vectors, we are able to magnify these vectors, considering only these objects, using the idea of the sequence minimization optimization.

(3) Lagrange Multiplier method for the support vector, we need certain constraint conditions.

Let's say the distance from our point to the super plane is D, and we ask for d>1 points as constraints.

The assumption that there is no such constraint can make the calculation error. This formula is the set of points that we go to the smallest point in the hyper-plane distance, and satisfies
。

The problem of finding the extremum under the condition of existence of constraint. We use the Lagrange multiplier method (see Baidu Encyclopedia).


(4) VariantsReference Lagrange formula F (x1,x2,... λ) =f (x1,x2,...) -λg (x1,x2 ...). Let's turn the above equation into
the constraints become
The argument of the upper type C makes the relaxation variable, because we see some of the red dots in the graph are divided into the range of Green Point, in order to consider such a problem, introduce a variable to control.

The main task of SVM is to calculate the number of references C.

"Machine Learning Algorithm-python implementation" SVM Support vector Machine (1)-Introduction to theoretical knowledge