Algorithm Learning notes: KNN theory Introduction

Reading objects: Understanding the indicator functions, understanding the concepts of training sets, test sets.

1. Introduction

KNN algorithm is a kind of supervised learning classification method. The so-called supervised learning and non-supervised learning refers to whether the training data has the label category, if there is supervised learning, if otherwise non-supervised learning. The so-called K-nearest neighbor algorithm, that is, given a training data set, the new input instance, in the training data set to find the nearest neighbor of the K instances, the K-instance of the majority of a class (weighted), the input instance is classified into this class.

The original nearest neighbor method was proposed by cover and Hart in 1968, and then theoretically in-depth analysis and research, one of the most important methods in non-parametric method, it nearest Neighbor Pattern in the paper A detailed description of the algorithm accuracy is given in classification, and the error rate of the nearest neighbor method is higher than that of Bayesian error rate. [no time to see, Mark, easy to read later]

KNN calculates the distance between the input instance and each training instance, when the training set is large, the computation is very time consuming, in order to improve the efficiency of KNN search, we can consider using special structure to store the training data to reduce the number of distance calculation. [see reference 1, this data is not very small, temporarily did not see]

2.k-nearest neighbor classification algorithm

Figure 1 from Reference 2

3.KNN features

The data preprocessing is important when the computation is large, the noise is sensitive, and the dimension of each sample attribute is very different.

Figure 2 from Reference 2

Resources:

[1] Methods of statistical learning, Hangyuan Li, p41-44

[2] Introduction to Data Mining (full version), Pang-ning Tan, Michael Steinbach, Vipin Kumar (Ming Fan, Fan Hongjian, etc.), p137-139

end!!

Algorithm Learning notes: KNN theory Introduction