Programmer Training Machine Learning SVM algorithm sharing

Http://www.csdn.net/article/2012-12-28/2813275-Support-Vector-Machine

absrtact: support vector Machine (SVM) has become a very popular algorithm. This paper mainly expounds how SVM works, and also gives some examples of using Python scikits library. As an algorithm for training machine learning, SVM can be used to solve classification and regression problems, and kernel trick technology is used to transform data, and then to find an optimal boundary in the possible output according to the conversion information.

"CSDN report" Support vector Machines (SVM) has become a very popular algorithm. In this article, Greg lamp simply explains how it works, and he gives a few examples of using the Python scikits library. All the code is available on GitHub, and Greg lamp will further elaborate on the details of using scikits and Sklearn. CSDN This technical article is compiled and organized:

What is SVM?

SVM is an algorithm for training machine learning that can be used to solve classification and regression problems, and also uses a technique called kernel trick to convert data, and then, based on these transformations, to find an optimal boundary in the possible output. In short, it's about doing some very complex data transformations and then calculating how to separate the user's data based on predefined labels or outputs.

What makes it so powerful?

Of course, for SVM, it is fully capable of classifying and returning. In this article, Greg Lamp focuses on how to classify using SVM, especially non-linear SVM or SVM using nonlinear kernels. Non-linear SVM means that the algorithm calculates a boundary that is not necessarily a straight line, and the advantage is that it captures more complex relationships between the set of data points without having to rely on the user to perform the difficult transformations themselves. The disadvantage is that the training time is much longer because of more computational capacity.

What is kernel trick?

Kernel trick docking The data you receive: type in some of the features you think are more obvious, sort them out, and output some data that you don't really know, which is like unlocking a DNA strand. You start by looking for the vector of the data, and then pass it on to kernel trick, and then continue to decompose and recombine until a larger data set is formed, and often the data you see is very difficult to understand. This is where the magic is, the extended data set has a more pronounced boundary, and the SVM algorithm can calculate a more optimized hyper-plane.

Second, suppose you are a farmer and now you have a problem-you need to build a fence to prevent wolves from harming the herd. But where should the fence be built? If you are a data-driven farmer, then you need to build a "classifier" on your ranch, based on the position of the herd and the wolves, and compare the different classifiers as shown, we can see that the SVM has completed a perfect solution. Greg Lamp sees the story as a beautiful illustration of the advantages of using a non-linear classifier. It is obvious that both the logical pattern and the decision tree pattern are using a straight line approach.

The implementation code is as follows: farmer.py Python

2. Import NumPy as NP
4. Import Pylab as Pl
6. From Sklearn import SVM
8. From Sklearn import Linear_model
10. From Sklearn import tree
12. Import Pandas as PD
18. def plot_results_with_hyperplane (CLF, Clf_name, DF, PLT_NMBR):
20. X_min, X_max = df.x.min ()-. 5, Df.x.max () +. 5
22. Y_min, Y_max = df.y.min ()-. 5, Df.y.max () +. 5
26. # Step between points. i.e. [0, 0.02, 0.04, ...]
28. step =.
30. # to plot the boundary, we ' re going to create a matrix of every possible point
32. # then label each point as a wolf or cow using our classifier
34. xx, yy = Np.meshgrid (Np.arange (x_min, X_max, step),
36. Np.arange (y_min, Y_max, Step))
38. Z = clf.predict (Np.c_[xx.ravel (), Yy.ravel ()])
40. # This gets we predictions back into a matrix
42. Zz = z.reshape (xx.shape)
46. # Create a subplot (we ' re going to the than 1 plot on a given image)
48. Pl.subplot (2, 2, PLT_NMBR)
50. # Plot the Boundaries
52. Pl.pcolormesh (xx, yy, Z, cmap=pl.cm.Paired)
56. # Plot the Wolves and cows
58. For animal in Df.animal.unique ():
60. Pl.scatter (df[df.animal==animal].x,
62. df[Df.animal==animal].y,
64. marker=Animal,
66. label="cows" if animal== "x" Else "Wolves",
68. color=' black ',
70. c=df.animal_type, cmap=pl.cm.Paired)
72. Pl.title (Clf_name)
74. Pl.legend (loc="best")
80. data = Open ("Cows_and_wolves.txt"). Read ()
82. data = [Row.split (' \ t ') for row in Data.strip (). Split (' \ n ')]
86. Animals = []
88. For Y, row in Enumerate (data):
90. For x, item in enumerate (ROW):
92. # x ' s is cows, O ' s is Wolves
94. If item in [' O ', ' x ']:
96. Animals.append ([x, Y, item])
100. DF = PD. DataFrame (Animals, columns=["x", "Y", "Animal"])
102. df[' animal_type ' = df.animal.apply (lambda x:0 if x== "x" Else 1)
106. # Train using the X and Y position coordiantes
108. Train_cols = ["X", "Y"]
112. Clfs = {
114. "SVM": SVM. SVC (),
116. "Logistic": Linear_model. Logisticregression (),
118. "Decision tree": Tree. Decisiontreeclassifier (),
120. }
124. PLT_NMBR = 1
126. For Clf_name, CLF in Clfs.iteritems ():
128. Clf.fit (Df[train_cols], Df.animal_type)
130. Plot_results_with_hyperplane (CLF, Clf_name, DF, PLT_NMBR)
132. PLT_NMBR + = 1
134. Pl.show ()

Let the SVM do some more difficult work!

Admittedly, if the relationship between the independent variable and the dependent variable is nonlinear, it is difficult to approach the accuracy of SVM. If it's still hard to understand, take a look at the following example: Suppose we have a set of datasets that contain green and red point sets. We first plot their coordinates, which form a concrete shape-with a red outline, surrounded by green (which looks like the flag of Bangladesh). If, for some reason, we lose 1/3 of the data set, then as we recover, we want to find a way to maximize the contours of this lost 1/3 part.

So how do we speculate that the missing 1/3 part is closest to what shape? One way is to create a model that uses nearly 80% of the data information as a "training set." Greg Lamp chooses three different data models to try out separately:

• Logical Model (GLM)
• Decision Tree Model (DT)
• Svm

Greg lamp trains each data model and then uses the models to speculate on the loss of 1/3 parts of the dataset. We can look at the results of these different models:

The implementation code is as follows: svmflag.py Python

2. Import NumPy as NP
4. Import Pylab as Pl
6. Import Pandas as PD
10. From Sklearn import SVM
12. From Sklearn import Linear_model
14. From Sklearn import tree
18. From Sklearn.metrics import Confusion_matrix
22. X_min, X_max = 0,
24. Y_min, Y_max = 0, ten
26. Step =. 1
28. # to plot the boundary, we ' re going to create a matrix of every possible point
30. # then label each point as a wolf or cow using our classifier
32. xx, yy = Np.meshgrid (Np.arange (x_min, X_max, Step), Np.arange (y_min, Y_max, Step))
36. DF = PD. DataFrame (data={' x ': Xx.ravel (), ' Y ': Yy.ravel ()})
40. df[' color_gauge ' = (df.x-7.5) **2 + (df.y-5) **2
42. df[' Color ' = df.color_gauge.apply (lambda x: "Red" if x <=-Else "green")
44. df[' color_as_int ' = df.color.apply (lambda x:0 if x== "red" Else 1)
48. Print "Points on flag:"
50. Print df.groupby (' color '). Size ()
52. Print
56. Figure = 1
60. # Plot a figure for the entire dataset
62. For color in Df.color.unique ():
64. idx = df. Color==color
66. Pl.subplot (2, 2, figure)
68. Pl.scatter (df[idx].x, df[idx].y, colorcolor=color)
70. Pl.title (' Actual ')
76. Train_idx = df.x <
80. Train = Df[train_idx]
82. Test = Df[-train_idx]
88. Print "Training Set Size:%d"% Len (train)
90. Print "Test Set Size:%d"% len (test)
94. # Train using the X and Y position coordiantes
96. cols = ["X", "Y"]
100. Clfs = {
102. "SVM": SVM. SVC (degree=0.5),
104. "Logistic": Linear_model. Logisticregression (),
106. "Decision tree": Tree. Decisiontreeclassifier ()
108. }
114. # racehorse different classifiers and plot the results
116. For Clf_name, CLF in Clfs.iteritems ():
118. Figure + = 1
122. # Train the classifier
124. Clf.fit (Train[cols], train.color_as_int)
128. # Get the predicted values from the test set
130. test[' predicted_color_as_int ' = Clf.predict (Test[cols])
132. test[' Pred_color ']
134. = Test.predicted_color_as_int.apply (lambda x: "Red" if x==0 Else "green")
138. # Create a new subplot on the plot
140. Pl.subplot (2, 2, figure)
142. # Plot each predicted color
144. For color in Test.pred_color.unique ():
146. # plot only rows where pred_color are equal to color
148. idx = test. Pred_color==color
150. Pl.scatter (test[idx].x, test[idx].y, colorcolor=color)
154. # Plot the training set as well
156. For color in Train.color.unique ():
158. idx = train. Color==color
160. Pl.scatter (train[idx].x, train[idx].y, colorcolor=color)
164. # Add a dotted line to show the boundary between the training and test set
166. # (Everything to the right of the "in the test set)
168. #this plots a vertical line
170. train_line_y = np.linspace (y_min, Y_max) #evenly spaced array from 0 to ten
172. train_line_x = np.repeat (ten, Len (train_line_y))
174. #repeat (threshold for Traininset) n times
176. # Add a black, dotted line to the subplot
178. Pl.plot (train_line_x, train_line_y, ' k--', color="Black")
182. Pl.title (Clf_name)
186. Print "Confusion Matrix for%s:"% clf_name
188. Print Confusion_matrix (Test.color, Test.pred_color)
190. Pl.show ()

Conclusion:

From these experimental results, there is no doubt that SVM is the absolute winner. But for the reasons, we might as well look at the DT model and the GLM model. It is clear that they are all using a straight line boundary. Greg Lamp's input model does not contain any conversion information when calculating the relationship between non-linear x, Y, and color. If Greg Lamp could define some specific conversion information so that the GLM model and the DT model would be able to output better results, why would they waste their time? In fact, there is no complex conversion or compression, SVM only analyzes the wrong 117/5000 points set (up to 98% accuracy, compared to the DT model is 51%, and GLM model only 12%!) ）

Where is the limitation?

Many people have doubts, since SVM is so powerful, but why can't we use SVM for everything? Unfortunately, the most magical part of SVM happens to be its biggest weakness! Complex Data transformation information and the results of boundary generation are difficult to elaborate. That's why it's often called black box, and the GLM model is just the opposite of the DT model, and they're easy to understand. (Compile/@CSDN Wang Peng, review/Zhonghao)

This article is compiled for CSDN, not reproduced without permission. If you want to reprint please contact [email protected]

Programmer Training Machine Learning SVM algorithm sharing