Machine learning Techniques (3)--kernel support Vector machines

Based on the last deduced question:

From a computational standpoint, if the dimension is too large, the inner product of Vector z is very time-consuming to solve.

We can split this process into two steps, first a conversion of X-space to Z-space, and then an inner product in Z-space. If you can combine these two steps to calculate a little faster, you can avoid this large amount of computation.

X and X ' converted to redo the inner product:

Such a method can be calculated faster, because directly in the X-space calculation is good, not in the Z-space calculation:

And such conversions are called kernel function:

How is this kernel used in SVM?

As can be seen, kernel trick is a method which avoids the calculation at high latitude space. Based on the substitution, the kernel SVM is derived:

Moreover, kernel SVM only needs SV when doing prediction.

Here are more kernel of other forms of two-item conversions:

The above retraction can be refined into a more general form:

As soon as we change the kernel slightly, the distance from the point to the line we are looking for will also change (Distance/margin), and the SV will change accordingly. So kernel also have to choose carefully.

From the 2-time term kernel, the kernel derivation of the higher-level item:

Regardless of the dimensions, the benefits of kernel are applicable:

Kernel when doing 1-dimensional conversions:

Such a simple conversion with the original method to solve the good.

What if this dimension is infinitely large?

It is proved that the kernel of infinite multidimensional is also feasible. So it leads to Gaussian kernel:

Put it into the SVM:

The linear function of the center on the SV is obtained. So the nature of Gauss kernel is:

Here's a comparison of the pros and cons of different kernel:

Linear kernel

Polynomial kernel

Gaussian kernel

Introduction to other Kernel:

Kernel represents the inner product of a vector, which is the similarity of vectors. Kernel must have two traits, one is symmetry and the other is semi-definite.

Summarize:

Machine learning Techniques (3)--kernel support Vector machines