SVM Support Vector Machine (2) simplified SMO algorithm for machine learning algorithm python implementation

Source: Internet
Author: User
Tags continue copy min range svm

1. Background knowledge

In the last section, we introduce the Lagrange boarding to obtain the support vector machine deformation formula. Detailed reform can refer to the Great God's blog-address

Reference to Lagrange formula F (x1,x2,... λ) =f (x1,x2,...) -λg (x1,x2 ...). We have transformed the above formula to:

The constraint becomes:

The following is based on the minimal optimization algorithm SMO (sequential minimal optimization). Find the nearest point of the distance divider, which is the support vector set. Shown in the blue dots of the following figure.

This column more highlights: http://www.bianceng.cnhttp://www.bianceng.cn/Programming/sjjg/

2. Code

Import Matplotlib.pyplot as PLT from numpy Import * to time import sleep def loaddataset (fileName): DA Tamat = [];  
        Labelmat = [] fr = open (fileName) for line in Fr.readlines (): Linearr = Line.strip (). Split (' t ') Datamat.append ([Float (linearr[0]), float (linearr[1])]) labelmat.append (float (linearr[2))) return DA  
        Tamat,labelmat def selectjrand (i,m): j=i #we want to select any J. equal to I (j==i):  
    j = Int (Random.uniform (0,m)) return J def Clipalpha (aj,h,l): if aj > H:aj = H  
    If l > Aj:aj = l return AJ def smosimple (Datamatin, Classlabels, C, Toler, Maxiter): Datamatrix = Mat (Datamatin); Labelmat = Mat (Classlabels). Transpose () b = 0; M,n = shape (datamatrix) Alphas = Mat (Zeros ((m,1))) iter = 0 while (ITER < Maxiter): Alphapair  
          schanged = 0 for I in range (m):  FXi = float (multiply (alphas,labelmat). T* (Datamatrix*datamatrix[i,:]. T) + b Ei = Fxi-float (Labelmat[i]) #if checks if an example violates KKT conditions if (lab  
                Elmat[i]*ei <-toler) and (Alphas[i] < C)) or ((Labelmat[i]*ei > Toler) and (alphas[i) > 0)): j = Selectjrand (i,m) FXJ = float (multiply (alphas,labelmat). T* (Datamatrix*datamatrix[j,:]. T)) + b Ej = Fxj-float (labelmat[j]) Alphaiold = Alphas[i].copy ();  
                Alphajold = Alphas[j].copy (); if (Labelmat[i]!= labelmat[j]): L = max (0, Alphas[j]-alphas[i]) H = min (C,  
                    C + alphas[j]-alphas[i]) else:l = max (0, Alphas[j) + alphas[i]-C) H = min (C, alphas[j] + alphas[i]) # if L==h:print "L==h"; Continue ETA = 2.0 * datamatrix[i,:]*datamatrix[j,:]. T-datamatrix[i,:]*daTamatrix[i,:]. T-datamatrix[j,:]*datamatrix[j,:]. T if ETA >= 0:print "eta>=0";                  Continue alphas[j]-= labelmat[j]* (Ei-ej)/eta alphas[j] = Clipalpha (alphas[j],h,l) # if (ABS (Alphas[j]-Alphajold) < 0.00001): print "J not moving Enough";  
                                                                        Continue alphas[i] + = labelmat[j]*labelmat[i]* (Alphajold-alphas[j]) #update I by the same amount as J  
                #the update is in the oppostie direction B1 = b-ei-labelmat[i]* (Alphas[i]-alphaiold) *datamatrix[i,:]*datamatrix[i,:]. t-labelmat[j]* (Alphas[j]-alphajold) *datamatrix[i,:]*datamatrix[j,:]. T b2 = b-ej-labelmat[i]* (Alphas[i]-alphaiold) *datamatrix[i,:]*datamatrix[j,:]. t-labelmat[j]* (Alphas[j]-alphajold) *datamatrix[j,:]*datamatrix[j,:]. T if (0 < alphas[i]) and (C > Alphas[i]): b = B1 elif (0 < AlphaS[J]) and (C > Alphas[j]): b = B2 Else:b = (B1 + b2)/2.0 alphapairschanged = 1 # Print "ITER:%d i:%d, pairs changed%d"% (iter,i,alphapairschanged) if (alphapairschanged = 0):  
      
ITER + + 1 else:iter = 0 # print "Iteration number:%d"% iter return B,alphas def matplot (datamat,lablemat): Xcord1 = []; Ycord1 = [] Xcord2 = []; Ycord2 = [] Xcord3 = [];  
           Ycord3 = [] for I in range (MB): if Lablemat[i]==1:xcord1.append (datamat[i][0) Ycord1.append (Datamat[i][1]) else:xcord2.append (datamat[i][0]) ycord2.append (Datamat I  
             [1]) B,alphas=smosimple (datamat,labelmat,0.6,0.001,40) for J in Range (MB): if alphas[j]>0: Xcord3.append (Datamat[j][0]) ycord3.append (datamat[j][1]) FIG = plt.figure ( ) ax = Fig.add_subplot (111) ax.scatter (Xcord1, Ycord1, s=30, c= ' Red ', marker= ' s ') Ax.scatter (Xcord2, Ycord2, s=30, c= ' green ') ) Ax.scatter (Xcord3, Ycord3, s=80, c= ' Blue ') Ax.plot () Plt.xlabel (' X1 ');  
    Plt.ylabel (' X2 ');  
Plt.show () If __name__== ' __main__ ': Datamat,labelmat=loaddataset ('/users/hakuri/desktop/testset.txt ') # b,alphas=smosimple (datamat,labelmat,0.6,0.001,40) # print b,alphas[alphas>0] Matplot (datamat,l Abelmat)

Author: csdn Blog Rae Garvin

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.