K-Nearest Neighbor algorithm (KNN)

1, K-Nearest neighbor algorithm principle

1.1 Algorithm Features

Simply put, the K-nearest neighbor algorithm uses the distance method of measuring different eigenvalues to classify.

Advantages: high accuracy, insensitive to outliers, no data input assumptions

Cons: High computational complexity, high spatial complexity

applicable data range: numerical and nominal type

1.2 Working principle

There is a training sample set, and each sample has a label (supervised learning). After entering new sample data without a label, each feature of the new data is compared to the feature in the sample set, and the algorithm extracts the most similar data (nearest neighbor) to the sample set feature. In general, we only select the first k most similar data in the sample data set, which is the source of K in the K-nearest neighbor algorithm, and K is usually not greater than 20. Finally, the most frequently occurring classification of K most similar data is selected as the classification of new data.

1.3 Example explanation

We use the K-nearest neighbor algorithm to classify love and action movies as an example of movie classification. Some have counted the fights and kissing footage of many movies, showing the number of fights and kissing shots in 6 movies. If there is a film not seen, how to determine whether it is a love movie or action movie?

① first need to count the number of fights and kissing scenes in this unknown movie, where the question mark position is how many shots the unknown movie appeared

After ② calculates the distance (similarity) between the unknown movie and the other movies in the sample set, the algorithm ignores the results as shown in the following table:

③ sort the similarity list to select the first k most similar samples. Here we assume that the first 3 k=3 of the similarity in the table above are: He's not really to dudes,beautiful Woman,california man.
④ Statistics The classification of the most similar samples. It's easy to know here that all 3 samples are love movies.
⑤ classifies the most categorized categories as unknown movies. Then we come to the conclusion that the unknown movie belongs to the love film.

2. Application of K-nearest neighbor simple classification

2.1 Algorithm General Flow

2.2 Python implementation code and annotations

#-*-coding:utf-8-*-ImportNumPy as NPdefCreateDataSet (): DataSet= Np.array ([[1,1,1,1], [2, 2, 2,3], [8, 8,8,9], [9, 9, 9,8]]) label= ['A','A','B','B']    returnDataSet, Labeldefclassify (input, dataSet, label, K): DataSize=dataset.shape[0] diff= Np.tile (input, (datasize, 1))-DataSet Sqdiff= diff * * 2squaredist= Np.sum (Sqdiff, Axis=1) Dist= squaredist**0.5Sortdistindex=Np.argsort (Dist) ClassCount= {}     forIinchRange (k): Votelabel=Label[sortdistindex[i]] Classcount[votelabel]= Classcount.get (Votelabel, 0) + 1Max=0 forKey, ValueinchClasscount.items ():ifMax <Value:max=Value Classes=KeyreturnClasses

Reference: http://www.cnblogs.com/hemiy/p/6155425.html

K-Nearest Neighbor algorithm (KNN)