Machine learning--k Nearest neighbor algorithm (k-nearest Neighbor,python implementation)

First, what is the KNN algorithm?
Second, the general flow of KNN algorithm
Three, the KNN algorithm Python code implementation

NumPy Module Reference Tutorial: http://old.sebug.net/paper/books/scipydoc/index.html

One: What is the look KNN algorithm?
KNN algorithm full name is K-nearest neighbor algorithm (K-nearest Neighbor)
The core idea of the KNN algorithm is that if the majority of the k nearest samples in a feature space belong to a category, the sample also falls into this category and has the characteristics of the sample on this category. This method determines the category to which the sample is to be divided, depending on the category of one or more adjacent samples in determining the classification decision.
Below is an example:

Even if you don't know what type of movie The unknown is, we can calculate it in some way, such as

Now that we have the distance between the sample set and the unknown movie, we can find the nearest movie in K distance according to the increment order of distance, assuming k=3, then the three closest movie is and he is not realy into dudes,beautiful women, California The man KNN algorithm determines the unknown movie type according to the nearest three movie types, these three are love movies, so the unknown type of movie is also love movie
Second: The general flow of KNN algorithm

STEP.1--- Initialize distance to maximum value

STEP.2--- calculates the distance between an unknown sample and each training sample dist

STEP.3--- get the current K The maximum distance maxdist in the nearest sample

STEP.4--- If Dist is less than maxdist, the training sample is used as the K-nearest neighbor sample

STEP.5--- Repeat step 2 , 3, 4, until the distance between the unknown sample and all training samples is counted.

STEP.6--- Statistics K - number of occurrences of each class label in the nearest neighbor sample

STEP.7--- Select the class label with the most frequent occurrence as the class label for the Unknown sample

Three, the KNN algorithm Python code implementation
Call mode: Open cmd, enter the directory where the knn.py file is located, enter Python, enter import KNN group,labels = Knn.createdataset () knn.classify0 ([0,0],group , lables,3)

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Machine learning--k Nearest neighbor algorithm (k-nearest Neighbor,python implementation)