Overview of SVM

Part 1 Introduction

 

Data-based machine learning is an important aspect of modern intelligent technology. It studies the laws from the perspective of observation data (samples) and uses these rules to predict future data or unobserved data. statistics is one of the important theoretical foundations of existing machine learning methods, including pattern recognition and neural networks. the traditional statistical research is based on this assumption that the number of samples tends to be infinite. however, in practice, the number of samples is often limited, so some theoretically excellent learning methods may not be satisfactory in practice. compared with traditional statistics, Statistical Learning Theory (SLT) is a Theory dedicated to studying the law of machine Learning in the case of small samples. vapnik and others have been devoted to this research since and. By the middle of, with the continuous development and maturity of their theories, due to the lack of substantial progress in theory in learning methods such as neural networks, the statistical learning theory has been paid more and more attention. statistical Learning Theory is based on a solid set of theories and provides a unified framework for solving the problem of Finite Sample learning. it can include many existing methods and is expected to help solve many difficult problems (such as Neural Network Structure Selection and Local Minimization). At the same time, based on this theory, a new general learning method-Support Vector Machine (SVM) has been developed. It has initially demonstrated many advantages over existing methods. some scholars believe that SLT and SVM are becoming new research hotspots after the study of neural networks, and will effectively promote the development of machine learning theories and technologies.

As early as the end of 1980s, some scholars noticed the basic results of the statistical learning theory, but less research later. Currently, only a few scholars have recognized this important research direction. no breakthroughs have been made in the key multi-classification SVM research in this paper.

 

 

Part 2 common classification technologies and Algorithms for Data Mining

 

 

1. Common techniques for Classified Data Mining

　

As a very important task in data mining, classification is currently the most widely used in business. The purpose of classification is to learn a classification function or classification model (also known as a classifier). This model can map data items in the database to one of the given classes for prediction. At present, there are many research results on classification methods, and the quality of discriminant methods can be achieved from three aspects: 1) prediction accuracy (accuracy of non-sample data); 2) computing complexity (the time and space complexity of the method implementation); 3) conciseness of the mode (in the same effect, the decision tree is expected to be smaller or fewer rules ).

In recent years, the research on Classification Algorithms in data mining has become a hot topic in this field. There are many comparative research achievements on different classification methods. No classification method is optimal for classification learning on all datasets. At present, the earliest and most widely used classification algorithm in data mining software is neural networks, which can quickly model non-linear data, adjust the network structure and connection weights of the training set through repeated learning, and classify and predict unknown data. However, in a sense, neural networks are a heuristic learning machine with a lot of experience. To overcome the inevitable difficulties of traditional neural networks, vapnik proposed a new neural network-support vector machine, and then proposed a statistical learning theory based on the Structure Risk Minimization idea, which formally laid the theoretical foundation of SVM, given the solid theoretical basis of SVM

 

2. Data Mining Classification Algorithms

(1) Discriminant Analysis

Linear Discriminant, KNN, Bayes discriminant, multivariate regression analysis, Rocchio method, distance function method, support vector machine, Potential Function Method

(2) machine learning

ID3 decision tree, AQ11 algorithm, Rough Sets

(3) Neural Networks

(4) Support Vector Machine

 

 

 

Part 3 Support Vector Machine

 

1. Overview of SVM

 

V. Vapnik's support vector machine theory has received wide attention in recent years due to its solid theoretical foundation and many excellent features. Many facts have proved that the structured Risk Minimization principle (SRM), one of the most basic concepts of SVM, is superior to the traditional Empirical Risk Minimization principle (Empirical Risk Minimization, ERM ). Unlike ERM's attempt to minimize the error in the training set, SRM tries to minimize the upper bound of the VC dimension so that it can achieve better promotion performance, this is precisely one of the most important goals of statistical learning theory. The main application fields of SVM include pattern recognition, function approximation, and Probability Density Estimation. This article focuses on the use of SVM for multiclass classification.

 

 

2. Support Vector Machines:

(1) Advantages of SVM:

The support vector machine method is based on the VC Dimension Theory of the Statistical Learning Theory and the minimum structure risk principle, based on the limited sample information, we can find the best compromise between the complexity of the model (that is, the learning Accuracy of specific training samples, Accuracy) and the learning ability (that is, the ability to identify arbitrary samples without error, in order to obtain the best promotion capability (Generalizatin Ability ). The main advantages of the SVM method are:

It can solve the problem of machine learning in the case of small samples.

Can improve generalization performance

It can solve high-dimensional problems

Can solve non-linear Problems

Avoid Neural Network Structure Selection and Local Minimization

 

(2) research hotspots of SVM

Currently, SVM algorithms are applied in many fields. For example, in terms of pattern recognition, SVM algorithms are more accurate than or equal to traditional learning algorithms for issues such as handwritten digital recognition, speech recognition, face image recognition, and Article classification. SVM has the following research hotspots:

Pattern Recognition

Regression Estimation

Probability Density Estimation

 

(3) Main kernel functions of SVM

Polynomial kernel: (gamma * U' * v + coef0) ^ degree

Radial Basis core (RBF): exp (-gamma * | u-v | ^ 2)

Sigmoid core: tanh (gamma * U' * v + coef0)

 

(4) Application of SVM

Text Classification and Face Recognition

3D object recognition and Remote Sensing Image Analysis

Function Approximation and time series prediction

Data Compression to optimize SVM algorithms

SVM improvement method (multiclass classification extension, used for multiclass classification problems)

SVM hardware implementation

 

(5) Difficulties in SVM

How to apply the Statistical Learning Theory (SLT) in unsupervised Pattern Recognition)

How to Use theoretical or experimental methods to calculate VC dimensions

The relationship between empirical risk and actual risk is called a promotional field. However, when (h/n)> 0.37 (h-VC Dimension, n-sample number ), the promotional realm is relaxed. How can we find a parameter that better reflects the learning capability of machines and obtain a tighter realm?

How to select a subset structure when minimizing structural risks (SRM)

 

(6) problems in applications

In addition to some of the key points discussed above, we need to solve the following problems:

(1) traditional SVM is used for binary classification. How can we expand to multiclass classification, such as predicting financial risks? If the risk level is high or low, traditional SVM can be used to solve the problem well. However, if one or more risk levels are added, they need to be extended into multi-classification SVM. There are a lot of research and few applications in this area.

(2) The computing performance of massive data is a problem faced by many data mining algorithms. Currently, SVM has to do a lot of research in this area.

This article from the CSDN blog, reproduced please indicate the source: http://blog.csdn.net/chl033/archive/2008/07/29/2729495.aspx