KNN algorithm python implementation and simple digital recognition methods, knn algorithm python Recognition

KNN algorithm python implementation and simple digital recognition methods, knn algorithm python Recognition

This article describes the implementation of kNN algorithm python and simple digital recognition methods. Share it with you for your reference. The details are as follows:

Advantages and disadvantages of kNN algorithm:

Advantages: high precision, insensitive to abnormal values, no input data assumptions
Disadvantage: both time complexity and space complexity are high.
Applicable data range: numeric and nominal

Algorithm ideas:

KNN algorithm (K-Nearest Neighbor Algorithm), the idea of the algorithm is very simple, simply put, it is a collection of things, that is, we find k from a bunch of known training sets closest to the target, then, let's see which of them is the most classification, and use this as the basis for classification.

Function parsing:

Library functions:

Tile ()
For example, tile (A, n) is to repeat A n times
Copy codeThe Code is as follows: a = np. array ([0, 1, 2])
Np. tile (a, 2)
Array ([0, 1, 2, 0, 1, 2])
Np. tile (a, (2, 2 ))
Array ([[0, 1, 2, 0, 1, 2], [0, 1, 2, 0, 1, 2])
Np. tile (a, (2, 1, 2 ))
Array ([[[0, 1, 2, 0, 1, 2], [[0, 1, 2, 0, 1, 2])
B = np. array ([[1, 2], [3, 4])
Np. tile (B, 2)
Array ([[1, 2, 1, 2], [3, 4, 3, 4])
Np. tile (B, (2, 1 ))
Array ([[1, 2], [3, 4], [1, 2], [3, 4])'
Self-implemented Functions

CreateDataSet () to generate a test Array
KNNclassify (inputX, dataSet, labels, k) classification function

InputX input parameters
DataSet training set
Label of the labels training set
Number of k nearest neighbors

Copy codeThe Code is as follows:
# Coding = UTF-8
From numpy import *
Import operator

Def createDataSet ():
Group = array ([[1.0, 0.9], [1.0, 1.0], [0.1, 0.2], [0.0, 0.1])
Labels = ['A', 'A', 'B', 'B']
Return group, labels
# InputX indicates the input vector (that is, we need to determine which class it belongs)
# DataSet indicates the training sample
# Label indicates the training sample label
# K is the nearest neighbor parameter. Select the nearest k
Def kNNclassify (inputX, dataSet, labels, k ):
DataSetSize = dataSet. shape [0] # several training data are calculated.
# Start to calculate Euclidean distance
DiffMat = tile (inputX, (dataSetSize, 1)-dataSet

SqDiffMat = diffMat ** 2
SqDistances = sqDiffMat. sum (axis = 1) # add each vector in the Matrix
Distances = sqDistances ** 0.5
# Euclidean distance calculated
SortedDistance = distances. argsort ()
ClassCount = {}
For I in xrange (k ):
VoteLabel = labels [sortedDistance [I]
ClassCount [voteLabel] = classCount. get (voteLabel, 0) + 1
Res = max (classCount)
Return res

Def main ():
Group, labels = createDataSet ()
T = kNNclassify ([0, 0], group, labels, 3)
Print t

If _ name __= = '_ main __':
Main ()

KNN application instance

Implementation of the handwriting recognition system

Dataset:

Two datasets: training and test. The classification label is in the file name. Pixel 32*32. The data looks like this:

Method:

KNN is used, but this distance is complicated (1024 features). It is mainly used to solve the problem of how to read data. It can be called directly in comparison.

Speed:

The speed is still relatively slow. Here the dataset is: training 2000 +, test 900 + (i5 CPU)

When k = 3, 32 s + is required.

Copy codeThe Code is as follows:
# Coding = UTF-8
From numpy import *
Import operator
Import OS
Import time

Def createDataSet ():
Group = array ([[1.0, 0.9], [1.0, 1.0], [0.1, 0.2], [0.0, 0.1])
Labels = ['A', 'A', 'B', 'B']
Return group, labels
# InputX indicates the input vector (that is, we need to determine which class it belongs)
# DataSet indicates the training sample
# Label indicates the training sample label
# K is the nearest neighbor parameter. Select the nearest k
Def kNNclassify (inputX, dataSet, labels, k ):
DataSetSize = dataSet. shape [0] # several training data are calculated.
# Start to calculate Euclidean distance
DiffMat = tile (inputX, (dataSetSize, 1)-dataSet
# DiffMat = inputX. repeat (dataSetSize, aixs = 1)-dataSet
SqDiffMat = diffMat ** 2
SqDistances = sqDiffMat. sum (axis = 1) # add each vector in the Matrix
Distances = sqDistances ** 0.5
# Euclidean distance calculated
SortedDistance = distances. argsort ()
ClassCount = {}
For I in xrange (k ):
VoteLabel = labels [sortedDistance [I]
ClassCount [voteLabel] = classCount. get (voteLabel, 0) + 1
Res = max (classCount)
Return res

Def img2vec (filename ):
ReturnVec = zeros ))
Fr = open (filename)
For I in range (32 ):
LineStr = fr. readline ()
For j in range (32 ):
ReturnVec [0, 32 * I + j] = int (lineStr [j])
Return returnVec

Def handwritingClassTest (trainingFloder, testFloder, K ):
HwLabels = []
TrainingFileList = OS. listdir (trainingFloder)
M = len (trainingFileList)
TrainingMat = zeros (m, 1024 ))
For I in range (m ):
FileName = trainingFileList [I]
FileStr = fileName. split ('.') [0]
ClassNumStr = int (fileStr. split ('_') [0])
HwLabels. append (classNumStr)
TrainingMat [I,:] = img2vec (trainingFloder + '/' + fileName)
TestFileList = OS. listdir (testFloder)
Error COUNT = 0.0
MTest = len (testFileList)
For I in range (mTest ):
FileName = testFileList [I]
FileStr = fileName. split ('.') [0]
ClassNumStr = int (fileStr. split ('_') [0])
VectorUnderTest = img2vec (testFloder + '/' + fileName)
ClassifierResult = kNNclassify (vectorUnderTest, trainingMat, hwLabels, K)
# Print classifierResult, '', classNumStr
If classifierResult! = ClassNumStr:
ErrorCount + = 1
Print 'tatal error', errorCount
Print 'error rate', errorCount/mTest

Def main ():
T1 = time. clock ()
HandwritingClassTest ('trainingdigits ', 'testdigits', 3)
T2 = time. clock ()
Print 'execute ', t2-t1
If _ name __= = '_ main __':
Main ()

I hope this article will help you with Python programming.