Kernel function __SVM of SVM series

The reason that kernel SVM actually hinders the above two kinds of SVM is that we have to take x through the mapping function in the case of non-linear processing Xphi x maps to Z domain because it is dealing with nonlinearity, it is to map the low dimension to high, but the characteristic of the mapped variable will increase rapidly sometimes it may reach infinity a bit extreme but the infinite variable through the kernel function can also be solved due to the very many characteristics so YNWTX1Y_NWT Phi x1 The computational capacity of linear Svmztzztz dual SVM is very large, very large. Common kernel function polynomial kernel function linear kernel function of Gaussian kernel function three kernel linear kernel gauss kernel with polynomial kernel

kernel SVM

Let's review the dual issue of the previous one.
Limitations of linear SVM

due to linear SVM final two Solver QP complexity relies on d+1 d+1 variables and n conditions. When using non-linear data, the X field is mapped to the Z field. The dimension of the Z-domain feature must be higher than X. Because you're dealing with nonlinearity, you must map low dimensional data to high-dimensional data (x maps to Z). Sometimes, this mapping, the feature will increase a lot, and even can reach the infinite dimension of the situation. Then the d+1 d+1 can be increased to infinity. Then, when we use linear SVM to deal with Non-linear and high-dimensional data, we can calculate the time to the everlasting.
SVM for dual forms
to solve the problem of linear SVM, we want to design a SVM so that he does not depend on the number d of the feature variables in the z domain after conversion.
So we consider using a dual approach to solve this problem.
The result we end up with is

This doesn't look like linear SVM, there is no definite z-domain variable Φ (x) \phi (x), but in fact when we throw the formula into the two-plan software, the computation of the QD Q_d he calculates is very much D˜\wideti Effect of Lde{d}. Because the calculation formula for each element of the QD q_d is

, this actually uses z field variable Z. So when the mapping, the features become a lot, then the Ztz z^tz computational volume will be very large. In fact, the dual form of SVM is still considered the z-domain variable z feature number, that is, consider the d˜\widetilde{d}. This is actually the same as linear SVM. That is, the reason that actually hinders the above two kinds of SVM is because, in dealing with nonlinearity, we map x through the mapping function Φ (x) \phi (x) to the Z field. Because it is dealing with nonlinearity, it is mapping a low dimension to a high level. However, after mapping, the characteristics of the variable will quickly increase, sometimes can reach infinity (a bit extreme, but through the kernel function, even infinite variables can be solved). Because of the very large number of features, then yn (wtφ (x) +b) >=1 y_n (w^t \phi (x) +b) >=1 (linear SVM), Ztz Z^tz (dual SVM), the two computational volume will be very high. Very big. Very big.

Some people ask, "Mapping X to Z, can the feature increase so much?"
Let's assume that X's dimension is D, and when we map it to the Z field, we just map the 1-second square to the 2-second side, and the resulting feature is
　
A lot of it. and then the corresponding multiplication and then add to the calculated amount is very complex.
　
So we are now using the kernel function to really solve the z domain feature problem.

We now solve the problem of large computational computation in dual functions.
For ease of expression, we still take the example of mapping to 2 times.
　　
Can actually be reduced to
　　
Then we just compute the inner product of the small x, and then we take the inner product of X, and after a multiplication of 2 additions, the inner product of Φ (x) \phi (x) is represented.
X is a D-dimensional, so the complexity is O (d), but Φ (x) \phi (x) has an internal product complexity of O (D2) O (d^2)
So we're going to define the kernel function K for this example
Not only can we use kernel function K to simplify qm,n q_{m,n}, but also can simplify the solution of B and the inner product with W, specifically for the following three kinds of cases can be reduced
　　
This replaces all occurrences of the z-domain variable with the kernel function K (x,x′) K (x,x '). The number of characters in the Z field is avoided.
The simplification of the above and its complexity can be summarized as
　
Then kernel SVM effectively avoids the Z-domain freedom