Machine learning actual Combat reading Notes (ii) K-Nearest neighbor algorithm

KNN algorithm:

1. Advantages: High precision, insensitive to outliers, no data input assumptions

2. Disadvantages: High computational complexity and high spatial complexity.

3. Applicable data range: Numerical and nominal type.

General Flow:

1. Collecting data

2. Preparing the data

3. Analyze data

4. Training algorithm: Not applicable

5. Test algorithm: Calculate the correct rate

6. Use algorithm: Need to input sample and structured output results, and then run the K-nearest neighbor algorithm to determine which classification of the input data, and finally applied to the computed classification to perform subsequent processing.

2.1.1 Importing data

operator is used when sorting.

 from Import *import  operatordef  CreateDataSet ():    Group=array ([[ 1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])    labels=['a','A  ','b','b']     return Group,labels

Save to knn.py file

Change current working directory, import KNN

Os.chdir ('g:\\ learning \ Machine learning combat ')import KNN

Call KNN to create a dataset

Group,labels=knn.createdataset ()

2.1.2 Implementation KNN algorithm

1. Calculate the distance between the points in the known category DataSet and the current point

2. Order by distance increment number order

3. Select the K points with the minimum distance from the current point

4. Determine the frequency of the category where the first K points are present

5. Return to the category with the highest frequency of the first K points as the current point of the forecast classification

4 Parameters:

A.inx: Input vectors for classification

B.dataset: Training Sample

C. Tag vector: Labels

D.K: Used to select the number of nearest neighbors

defclassify0 (InX, DataSet, labels, k): Datasetsize=Dataset.shape[0] Diffmat= Tile (InX, (datasetsize,1))-DataSet Sqdiffmat= Diffmat**2sqdistances= Sqdiffmat.sum (Axis=1) Distances= sqdistances**0.5sorteddistindicies=distances.argsort () ClassCount={}               forIinchRange (k): Voteilabel=Labels[sorteddistindicies[i]] Classcount[voteilabel]= Classcount.get (voteilabel,0) + 1Sortedclasscount= Sorted (Classcount.iteritems (), Key=operator.itemgetter (1), reverse=True)returnSORTEDCLASSCOUNT[0][0]

To calculate Euclidean distance

6 line by small to large sort distances.argsort (), after the order is subscript

2.2 Using KNN algorithm to improve the pairing effect of dating sites

adding functions in knn.py

Strip is to remove the back and forth of the \n,[-1] actually refers to the last column

defFile2matrix (filename): Fr=open (filename) numberoflines= Len (Fr.readlines ())#get The number of lines in the fileReturnmat = Zeros ((numberoflines,3))#prepare matrix to returnClasslabelvector = []#Prepare labels returnFR =open (filename) index=0 forLineinchfr.readlines (): line=Line.strip () listfromline= Line.split ('\ t') Returnmat[index,:]= Listfromline[0:3] Classlabelvector.append (listfromline[-1]) Index+ = 1returnReturnmat,classlabelvector

Reload the KNN and call the function

Reload (KNN) datingdatamat,datinglabels=knn.file2matrix ('datingTestSet.txt' )

2.2.2 Analyzing data: Creating a scatter plot with matplotlib

Import matplotlib Import Matplotlib.pyplot as Pltfig=plt.figure () Ax=fig.add_subplot (111) Ax.scatter (datingdatamat[:, 1],datingdatamat[:,2]) plt.show ()

Change the colors to show the different categories

Import matplotlib Import Matplotlib.pyplot as Pltfig=plt.figure () Ax=fig.add_subplot (111) Ax.scatter ( datingdatamat[:,1],datingdatamat[:,2],15.0*numpy.array (datinglabels), 15.0*Numpy.array (datingLabels) ) Plt.show ()

2.2.3 Preparing data: Normalized values

def Autonorm (dataSet):     = dataset.min (0)    = Dataset.max (0)    = maxvals- minvals    = zeros (Shape ( DataSet    ) = Dataset.shape[0]    = Dataset-tile (Minvals, (m,1))    = Normdataset/tile (ranges, (m,1))   #element wise divide    return Normdataset , Ranges, Minvals

2.2.4 as a complete program validation classifier

defdatingclasstest (): HoRatio= 0.50#Hold out 10%Datingdatamat,datinglabels = File2matrix ('DatingTestSet2.txt')#Load Data setfrom fileNormmat, ranges, minvals =autonorm (Datingdatamat) m=Normmat.shape[0] Numtestvecs= Int (m*hoRatio) Errorcount= 0.0 forIinchRange (numtestvecs): Classifierresult= Classify0 (normmat[i,:],normmat[numtestvecs:m,:],datinglabels[numtestvecs:m],3)        Print "The classifier came back with:%d, the real answer is:%d"%(Classifierresult, datinglabels[i])if(Classifierresult! = Datinglabels[i]): Errorcount + = 1.0Print "The total error rate is:%f"% (errorcount/float (numtestvecs))PrintErrorcount

Machine learning actual Combat reading Notes (ii) K-Nearest neighbor algorithm