ZZ collects kernel functions-kernel function

Ref

Http://www.zhizhihu.com/html/y2010/2292.html

Kernel Functions

Below is a list of some kernel functions available from the existing literature. As is the case with previous articles, every LaTeX notation for the formulas below is readily available from their A Lternate text HTML tag. I can not guarantee all of them is perfectly correct, thus use them at your own risk. The most of them has links to articles where they has been originally used or proposed.

1. Linear Kernel

The Linear kernel is the simplest kernel function. It is given by the inner product <x,y> plus an optional constant c. Kernel algorithms using a linear Kernel are ofte n equivalent to their Non-kernel counterparts, I.E.KPCA with linear kernel are the same as standard PCA.

2. Polynomial Kernel

The polynomial kernel is a non-stationary kernel. Polynomial kernels is well suited for problems where all the training data is normalized.

Adjustable parameters is the slope alpha, the constant term C and the polynomial degree d.

3. Gaussian Kernel

The Gaussian kernel is an example of radial basis function kernel.

Alternatively, it could also be implemented using

The adjustable parameter sigma plays a major role in the performance of the kernel, and should is carefully tuned to the P Roblem at hand. If overestimated, the exponential would behave almost linearly and the higher-dimensional projection would start to lose its Non-linear power. In the other hand, if underestimated, the function would lack regularization and the decision boundary would be highly sensi tive to noise in training data.

4. Exponential Kernel

The exponential kernel is closely related to the Gaussian kernel and with the square of the norm left out. It is also a radial basis function kernel.

5. Laplacian Kernel

The Laplace Kernel is completely equivalent to the exponential Kernel, and except for being less sensitive for changes in the Sigma parameter. Being equivalent, it is also a radial basis function kernel.

It is important to note that the observations made about the Sigma parameter for the Gaussian kernel also apply to the EXP Onential and Laplacian Kernels.

6. ANOVA Kernel

The ANOVA kernel is also a radial basis function kernel, just as the Gaussian and Laplacian kernels. It is said to perform well in multidimensional regression problems (Hofmann, 2008).

7. Hyperbolic Tangent (Sigmoid) Kernel

The hyperbolic Tangent Kernel is also known as the Sigmoid Kernel and as the Multilayer Perceptron (MLP) Kernel. The Sigmoid Kernel comes from the neural Networks field, where the bipolar Sigmoid function was often used as an activation function for artificial neurons.

It is interesting to note the a SVM model using a sigmoid kernel function is equivalent to a two-layer, Perceptron neural Network. This kernel is quite popular for support vectors machines due to their origin from neural network theory. Also, despite being only conditionally positive definite, it had been found to perform well in practice.

There is adjustable parameters in the sigmoid kernel, the slope alpha and the Intercept constant C. A common value for Alpha was 1/n, where N is the data dimension. A more detailed study on sigmoid kernels can is found in the works by Hsuan-tien and Chih-jen.

8. Rational Quadratic Kernel

The Rational quadratic kernel is less computationally intensive than the Gaussian kernel and can be used as an alternative When using the Gaussian becomes too expensive.

9. Multiquadric Kernel

The Multiquadric kernel can be used in the same situations as the Rational quadratic kernel. As is the case with the Sigmoid kernel, it's also an example of a non-positive definite kernel.

Inverse Multiquadric Kernel

The inverse Multi quadric kernel. As with the Gaussian kernel, it results in a kernel matrix with full rank (Micchelli, 1986) and thus forms a infinite dime Nsion feature space.

Circular Kernel

The circular kernel comes from a statistics perspective. It is a example of an isotropic stationary kernel and are positive definite in R2.

Spherical Kernel

The spherical kernel are similar to the circular kernel, but are positive definite in R3.

Wave Kernel

The Wave kernel is also symmetric positive semi-definite (Huang, 2008).

Power Kernel

The Power kernel is also known as the (unrectified) triangular kernel. It is a example of scale-invariant kernel (SAHBI and Fleuret, 2004) and is also only conditionally positive definite.

. Log Kernel

The Log kernel seems to being particularly interesting for images, but was only conditionally positive definite.

Spline Kernel

The Spline kernel is given as a piece-wise cubic polynomial, as derived in the works by Gunn (1998).

However, what's it actually mean is:

With

B-spline (Radial Basis Function) Kernel

The B-spline kernel is defined on the interval [−1, 1]. It is given by the recursive formula:

In the work by Bart Hamers It's given by:

Alternatively, Bn can be computed using the explicit expression (Fomel, 2000):

Where x+ is defined as the truncated power function:

Bessel Kernel

The Bessel kernel is well known in the theory of function spaces of fractional smoothness. It is given by:

Where J is the Bessel function of first kind. However, in the Kernlab for R documentation, the Bessel kernel are said to be:

Cauchy Kernel

The Cauchy kernel comes from the Cauchy distribution (Basak, 2008). It is a long-tailed kernel and can being used to give long-range influence and sensitivity over the high dimension space.

Chi-Square Kernel

The chi-square kernel comes from the chi-square distribution.

Histogram intersection Kernel

The histogram intersection Kernel is also known as the Min Kernel and have been proven useful in image classification.

Generalized histogram intersection

The generalized histogram intersection kernel is built based on the histogram intersection kernel for image classification But applies in a much larger variety of contexts (Boughorbel, 2005). It is given by:

Generalized t-student Kernel

The generalized t-student Kernel have been proven to be a mercel Kernel, thus have a positive semi-definite Kernel matrix (Boughorbel, 2004). It is given by:

Bayesian Kernel

The Bayesian kernel could be given as:

where

However, it really depends on the problem being modeled. For more information, please see the work by Alashwal, Deris and Othman, in which they used a SVM with Bayesian kernels in The prediction of Protein-protein interactions.

Wavelet Kernel

The Wavelet kernel (Zhang et al, 2004) comes from Wavelet theory and is given as:

Where A and C are the wavelet dilation and translation coefficients, respectively (the form presented above is a simplific ation the original paper for details). A translation-invariant version of this kernel can is given as:

Where in both H (x) denotes a mother wavelet function. In the paper by Li Zhang, Weida Zhou, and Licheng Jiao, the authors suggests a possible h (x) as:

Which they also prove as an admissible kernel function.

Via

ZZ collects kernel functions-kernel function