SVM (SVR) algorithm for machine learning notes

After learning SVM, read so many other people's articles, it is time to sum up a wave of their own. The right to write a note for yourself to review it later.
PS: Combined with their own in the work process (I use the SVR to do stock prediction) used to write the knowledge, not very comprehensive, if some knowledge is not mentioned here readers can find their own.
1 Overview
Support Vector Machines (SVM) is a classification algorithm, but can also do regression, according to the input data can be different models (if the input tag is a continuous value of the regression, if the input tag is the classification value of SVC () to do the classification). In order to improve the generalization ability of the learning machine and realize the minimization of the experience risk and the confidence range by seeking the minimum structure risk, we can obtain the good statistic rule in the case of less statistic sample quantity. In layman's terms, it is a two-class classification model, whose basic model is defined as the most spaced linear classifier on the feature space, that is, the learning strategy of support vector machine is to maximize the interval, and finally can be transformed into a convex two-time programming problem solution.

Advantages and disadvantages of SVM algorithm:
SVM Algorithm Advantage Area
• Can be used for linear/non-linear classification or for regression;
• Low generalization error;
• Easy to interpret;
• Low computational complexity;
• Can solve high-dimensional problems;
Disadvantage Area
• Sensitivity to the selection of parameters and kernel functions;
• Primitive SVM is only better at dealing with two classification problems;
SVM belongs to the supervised learning algorithm, which supervises the learning process as shown in the figure:

2 Algorithm principle
2-1 finds a categorical hyper-plane in n-dimensional space, classifying the points on the space. The following figure is an example of a linear classification.

Figure 2-1-1 Example of linear classification
2-2. In general, the proximity of a point distance over a plane can be expressed as the degree of certainty or accuracy of categorical predictions. SVM is to maximize this interval value. And the point on the dotted line is called the support vector Supprot Verctor. As shown in Figure 4-1-1-2:

Figure 2-1-1-2 Support vector identification diagram
2-3. In practice, we often encounter linear irreducible samples, at this time, our common practice is to map sample features into the high-dimensional space as shown in Figure 4-1-1-3:

Figure 2-1-1-3 Processing method for linear non-divided data
3 Data Processing
Preprocessing.scale () Effect:
Scale () is used for scaling the original sample, and the range can be self-determined, typically [0,1] or [ -1,1].
The purpose of scaling is mainly
1) Prevent a feature from being too large or too small to play an unbalanced role in training;
2) in order to calculate the speed. Because in nuclear calculations, an internal product operation or an exp operation is used, unbalanced data can cause computational difficulties.

4 SVR parameter selection
For the SVM algorithm, we first import the SVR module in SKLEARN.SVM. SVR () is the SVM algorithm to do regression method (that is, the input tag is a continuous value of the time to use the method), through the following statement to determine the SVR mode (select the more important parameters to test. Randomly select a stock to start the relevant parameter selection test).
SVR = SVR (kernel= ' RBF ', C=1e3, gamma=0.01)

4-1. Kernel: The types of nuclear functions , commonly used are ' RBF ', ' linear ', ' poly ', as shown in Fig. 4-1-2-1, it is found that the function model has the best fitting effect when using the RBF parameter.

Figure 4-1-2-1 function Parameters "Linear", "RBF", "Poly" to the model Fit effect diagram "

4-2. C: Penalty factor . c Characterize how much you value outliers, and the greater the value of C, the less you want to lose them. When the value of C is large, the penalty of error classification increases, and the penalty for error classification decreases with C value hour. When C is larger, approaching infinity, it means that the existence of categorical errors is not allowed, the smaller the margin, the easier it is to fit; When C tends to 0 o'clock, it means that we no longer focus on whether the classification is correct, only the greater the margin, easy to fit. As shown in Fig. 4-1-2-2, it is found to be most suitable when using 1E3.

Figure 4-1-2-2 Diagram of parameter C vs. accuracy

4-3 Gamma: is ' RBF ', ' poly ' and ' sigmoid ' nuclear coefficients and the gamma value must be greater than 0. With the increase of gamma, there is a better classification effect for the test set, and the error of generalization is easy to fit. As shown in Fig. 4-1-2-3, the gamma=0.01 is found to be the most accurate.

Figure 4-1-2-3 Parameter gamma vs. accuracy diagram

The above is probably the whole process of building stock forecasting model, here is the emphasis on the algorithm of the role of each method and the main parameters of the meaning of the representative, we hope to help.