Support Vector Machine (SVM) algorithm analysis--Zhou Zhihua's Watermelon book Study

1. Linear can be divided into

For a data set:

If there is a hyper-planar x that can precisely divide the positive and negative samples in D into the sides of S, the hyper-plane is as follows:

Then the data set D is linearly divided, otherwise, it is not possible to be divided.

W is called the normal vector, which determines the direction of the super plane, and B is the displacement amount, which determines the distance between the super plane and the origin point.

The distance from any point in the sample space X to the super-plane X (the content of the chapter that can be used to review the space geometry in higher numbers) can be written as:

The following two-form training samples are called support vectors:

The sum of two heterogeneous support vectors (one equals +1, one equals 1) to the hyper-plane distance:

It's called "interval."

To find the maximum interval dividing the hyper plane, is to make the maximum:

Equivalent to:

This is the basic model of support vector machines.

Dual problem:

The Lagrangian function of the upper type can be written as:

which

The derivation of the parameters W and B can be:

The upper is brought into the Lagrangian function, and the W and B are eliminated to obtain the dual expression:

The solution of duality problem is accomplished by SMO algorithm:

Original paper Address: Http://www-ai.cs.uni-dortmund.de/LEHRE/SEMINARE/SS09/AKTARBEITENDESDM/FOLIEN/Joerg_Nitschke_Sequential_ Minimal_optimization.pdf

Support Vector Machine (SVM) algorithm analysis--Zhou Zhihua's Watermelon book Study