Classification algorithm--k-Proximity

K- Nearest classification method by calculating the distance between the target to be classified and the training sample, selecting the most recent training sample from the target distance to be classified, and determining the sample to be classified according to the category of the majority in the K-selected sampling example. There are many distance types, roughly European distance, Manhattan distance, Chebyshev distance, min Koffsky distance, standardized European distance, Markov distance, angle cosine, hamming distance, correlation coefficient, information entropy, etc.

The specific steps of the KNN algorithm:

1. Calculate the distance dist between the target to be classified and each training sample;

2, the Dist to order, select the first k training sample as the K-nearest example;

3. Statistic the frequency of each category in K training samples;

4. Select the category with the most frequent frequency as the category of target to be classified.

According to the above process, the implementation code is given:

FROM NUMPY IMPORT *IMPORT OPERATORDEF KNN (x,transet,labels,k,type = 0):      '     k-nearest neighbor algorithm     :p aram x: Features to classify      :p aram transet:  known features     :p aram labels:  feature labels      :p aram k:    :p aram type:          distance Type: 0: European, 1: Manhattan, 2: Chebyshev, 3: Min Koffsky, 4: Standardized European, 5: MA, 6: Angle cosine                    7: Hamming distance, 8: Jaccard, 9: Correlation coefficient, 11: Information entropy      :return:     "    distances = zeros_like (labels)      if type == 0:        transize =  shape (Transet)         diffmat = tile (x, (tranSize[0],1) )  - transet        distances =  ((diffmat**2). SUM (Axis=1)) **0.5      #TODO: Calculates the distance     sortedindex = distances.argsort () according to the different types      CLASSCOUNT = {}    FOR I IN LABELS[SORTEDINDEX[:K]]:         classcount[i] = classcount.get (i,0)  + 1     sortedclasscount = sorted (Classcount.items (), Key=operator.itemgetter (1), Reverse=true)     return sortedClassCount;

The advantages of KNN algorithm: simple theory and simple implementation.

The disadvantage of KNN algorithm: 1) The choice of K-value depends on the large, different K-value selection may produce different results. The k value with the smallest classification error can be selected experimentally. 2) due to the need to calculate the distance of features, it is necessary to quantify and standardize features.

This article from "Go one stop two look back three" blog, please make sure to keep this source http://janwool.blog.51cto.com/5694960/1895064

Classification algorithm--k-Proximity