K相鄰演算法

標籤：number   operator   ever   --   list   element   and   app   names   

剛開始學習機器學習,先跟這《機器學習實戰》學一些基本的演算法

----------------------------------分割線--------------------------------------------

該演算法是用來判定一個點的分類,首先先找到離該點最近的k個點,然後找出這k個點的哪種分類出現次數最多,就把該點設為那個分類
距離公式選用歐式距離公式:

 

下面給出例子(來自《機器學習實戰》)

 

1.約會對象喜歡程度的判定:

現需要一個約會對象喜歡程度的分類器

給定資料集,屬性包含

每年獲得的飛行常客裡程數,玩視頻遊戲所耗時間百分比,每周消費的冰淇淋公升數,是否喜歡且喜歡程度(不喜歡,一般喜歡,特別喜歡),四個屬性

 

分類器代碼:

先擷取整個資料集的約會對象個數t,然後把我們要分類的矩陣複製t次

然後減去原資料集,得到xA0-xB0和xA1-xB1,然後對矩陣每個元素平方,行內求和,開根號,得到所有點的距離

統計前k個資料類別出現次數,返回次數最多的類別

 1 def classify0(inX, dataSet, labels, k): 2     dataSetSize = dataSet.shape[0] 3     diffMat = tile(inX, (dataSetSize,1)) - dataSet 4     sqDiffMat = diffMat**2 5     sqDistances = sqDiffMat.sum(axis=1) 6     distances = sqDistances**0.5 7     sortedDistIndicies = distances.argsort() 8     classCount={} 9     for i in range(k):10         voteIlabel = labels[sortedDistIndicies[i]]11         classCount[voteIlabel] = classCount.get(voteIlabel,0) + 112     sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True)13     return sortedClassCount[0][0]

先從檔案中分別讀前三個屬性和喜歡程度到矩陣

然後進行歸一化,最後分類得到結果統計一下就好了

 1 def file2matrix(filename): 2     fr = open(filename) 3     numberOfLines = len(fr.readlines())         #get the number of lines in the file 4     returnMat = zeros((numberOfLines,3))        #prepare matrix to return 5     classLabelVector = []                       #prepare labels return 6     fr = open(filename) 7     index = 0 8     for line in fr.readlines(): 9         line = line.strip()10         listFromLine = line.split(‘\t‘)11         returnMat[index,:] = listFromLine[0:3]12         classLabelVector.append(int(listFromLine[-1]))13         index += 114     return returnMat,classLabelVector15 16 def autoNorm(dataSet):17     minVals = dataSet.min(0)18     maxVals = dataSet.max(0)19     ranges = maxVals - minVals20     normDataSet = zeros(shape(dataSet))21     m = dataSet.shape[0]22     normDataSet = dataSet - tile(minVals, (m,1))23     normDataSet = normDataSet/tile(ranges, (m,1))   #element wise divide24     return normDataSet, ranges, minVals25 26 def datingClassTest():27     hoRatio = 0.50      #hold out 10%28     datingDataMat,datingLabels = file2matrix(‘datingTestSet2.txt‘)       #load data setfrom file29     normMat, ranges, minVals = autoNorm(datingDataMat)30     m = normMat.shape[0]31     numTestVecs = int(m*hoRatio)32     errorCount = 0.033     for i in range(numTestVecs):34         classifierResult = classify0(normMat[i,:],normMat[numTestVecs:m,:],datingLabels[numTestVecs:m],3)35         print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, datingLabels[i])36         if (classifierResult != datingLabels[i]): errorCount += 1.037     print "the total error rate is: %f" % (errorCount/float(numTestVecs))38     print errorCount

 

2.數字識別

給定訓練集包含32*32的01字串,然後判定測試集的數字

對於把32*32每個數字都看作一個屬性,然後直接算距離分類...

 1 def img2vector(filename): 2     returnVect = zeros((1,1024)) 3     fr = open(filename) 4     for i in range(32): 5         lineStr = fr.readline() 6         for j in range(32): 7             returnVect[0,32*i+j] = int(lineStr[j]) 8     return returnVect 9 10 def handwritingClassTest():11     hwLabels = []12     trainingFileList = listdir(‘digits/trainingDigits‘)           #load the training set13     m = len(trainingFileList)14     trainingMat = zeros((m,1024))15     for i in range(m):16         fileNameStr = trainingFileList[i]17         fileStr = fileNameStr.split(‘.‘)[0]     #take off .txt18         classNumStr = int(fileStr.split(‘_‘)[0])19         hwLabels.append(classNumStr)20         trainingMat[i,:] = img2vector(‘digits/trainingDigits/%s‘ % fileNameStr)21     testFileList = listdir(‘digits/testDigits‘)        #iterate through the test set22     errorCount = 0.023     mTest = len(testFileList)24     for i in range(mTest):25         fileNameStr = testFileList[i]26         fileStr = fileNameStr.split(‘.‘)[0]     #take off .txt27         classNumStr = int(fileStr.split(‘_‘)[0])28         vectorUnderTest = img2vector(‘digits/testDigits/%s‘ % fileNameStr)29         classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)30         print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr)31         if (classifierResult != classNumStr): errorCount += 1.032     print "\nthe total number of errors is: %d" % errorCount33     print "\nthe total error rate is: %f" % (errorCount/float(mTest))

 

K相鄰演算法