Nearest neighbor classifier (KNN)

First, Concept & significance

Find and test all training samples that are relatively close to the sample properties.

Using the most recent pro to determine the rationality of the class label, with the following words to best illustrate:

"If you walk like a duck, and you look like a duck, it's probably a duck," he said.

Second, the calculation steps:

1. Distance: Given the test object, calculate its distance from each object in the training set

2, looking for neighbors: delimit the nearest K-object, as the nearest neighbor of the test object

3, do classification: according to the K nearest neighbor attribution of the main category, to the test object classification

三、三个 Basic Elements

I, distance measurement

a) When p=2, for European-style distances:

B) When P=1, for Manhattan Distance:

II, Selection of K-values

The smaller the K value, the more complex the whole model is, and the easier it is to fit.

The larger the K value, the simpler the overall model, and the approximate error will increase (mis-classification)

Iii. Determination of test object categories

A) majority vote:

where V is the class designator, Yi is a nearest neighbor class designator,I(.) is the indicator function, if its argument is true, returns 1, otherwise, 0

b) Distance weighted vote:

Nearest neighbor classifier (KNN)