Course IntroductionTo continue with the last lesson, when the data is non-linear, you need to convert the data into Z space (linearly) to use the SVM, so you need to know what Z space is. This lesson solves the need to know the specific Z space can be used to classify SVM.Finally, the course describes how to fit the problem. The idea is the same as the one introduced in Lesson 11, which is to set a restrictive condition.
Course Outline:1, the kernel trick.2. Soft-margin SVM
1. The Kernel TrickAs you can tell by 14 lessons, we only need the Z-space to provide an inner product of two points, and we don't need to know how to convert X to Z.The process of solving the SV as described in.
Therefore, our goal is to find the following function:
z*z ' = K (x,x '); --the Kernel。As long as Z is real (but we don't need to know exactly what it is), the K function is valid. Maybe someone would ask, "Z doesn't exist, does it matter?" As long as you can find K so that for each pair of x,x ' can find the corresponding value. Perhaps in some cases, Z does not exist can also get the desired results, but because our derivation process is based on the existence of Z-case, so in order to ensure the correctness of training, we need to know that Z is present. Here's how to verify that a kernel function exists:1, it is symmetrical, should have: K (x,x ') = k (x ', x)2. Matrix:It's positive semi-definite.As long as K satisfies the above two conditions, it can be proved that Z exists.The remaining problem is to find a suitable K. The following two are commonly used:1) The polynomial kernelK (x,x ') = (ax*x ' +b) ^qwhich2) The RBF kernel (more commonly used in SVM)K (x,x ') = exp (-y| | X-x ' | | ^2)
With the core function, we don't have to design the conversion function ourselves, but we need to find a suitable kernel function.
2. Soft-margin SVMAs we mentioned in the 11th lesson, if the sample error is 0, then we are likely to calculate the noise, which will result in the reduced generalization ability of the learning model. So we should allow the presence of noise.So we need a way to measure the error.To illustrate the error measurement method used here, we first observe that the point in the red line is called the break point (which destroys the margin). Set its distance to margin of Q. We use Q as the destructive force of the yellow dots in the graph.The total destructive power is defined as: all the points in the yellow and the distance of the margin.After formulation there are:
So now we need
For an explanation of the first formula:
The beginning is not the sum of the following, when we minimize the previous part, the equivalent of maximizing: margin. Now we have added the following and modified constraints, and when we increase the margin we reduce the value of the first item while increasing the value of the second item, so these two items can be mutually constrained. The SV can be learned in case the noise is allowed to exist.
C in the first item is a constant term used to adjust the weight of the error. When C = infinity, it degenerates into the last lesson. There is no point in learning when C = 0. Because then the margin will become infinitely large.
The next task is to solve the above equation. The specific method can be referred to below.
Finally, there are two issues to note:1, if the data is linearly non-divided.When the data is linearly non-divided, we can also use the above method, but will come to an unacceptable solution, at this time we can detect whether the solution is valid to determine whether our data can be divided.2. What happens if W0 exists in Z?In our previous assumptions, W0 represents a constant term of 1, but when Z also exists W0, we make the constant item W0-B. When the study is complete, there will be:(Why?) ）
California Institute of Technology Open Class: machine learning and data mining _kernal Method (15th lesson)
