KNN Neighbor Algorithm

KNN algorithm decision-making process

K-Nearest Neighbor algorithm
　　

In the picture on the right, the Green Circle is determined to be assigned to which class, is it a red triangle or a blue square? If K = 3, because the proportion of the red triangle is 2/3, the green circle will be assigned to the class of the Red Triangle. If K = 5, because the proportion of the blue square is 3/5, therefore, the Green Circle is given a blue square category.
　

K-Nearest
Neighbor, KNN) classification algorithm is a theoretically mature method and one of the simplest machine learning algorithms. The idea of this method is: if a sample has k elements in the feature space
If most of the samples (that is, the most adjacent in the feature space) belong to a specific category, the sample also belongs to this category. In KNN algorithm, the selected neighbors are objects that have been correctly classified. This method is used for determining the class
The category of the samples to be classified is determined only based on the class of the nearest one or several samples.
The KNN method also relies on the Limit Theorem in principle, but in classification decision-making, it is only related to a very small number of adjacent samples. Since the KNN method mainly relies on a limited number of adjacent samples, rather than the method used to determine the similarity
The KNN method is more suitable than other methods for determining the category of a class.

KNN can be used for classification and regression. You can obtain the attributes of a sample by finding k nearest neighbors and assigning the average values of these neighbor attributes to the sample. A more useful method is to give different weights to the impact of different distance neighbors on the sample, for example, the weights are proportional to the distance.
　　

The main disadvantage of this algorithm in classification is that when samples are unbalanced, for example, the sample size of a class is large while that of other classes is large.
Very small, it is possible that when a new sample is input, a large number of samples in the K-neighbor capacity class of the sample account for the majority.
This algorithm only calculates "nearest" neighbor samples. A certain type of samples has a large number, so either this type of samples is not close to the target sample, or this type of samples is very close to the target sample. In any case, the quantity does not affect the operation.
Result. You can use the method of weight (a large neighbor weight with a small distance from the sample) to improve. Another disadvantage of this method is that it requires a large amount of computing, because each text to be classified must be calculated to all known samples.
To obtain K Nearest Neighbor points. Currently, the common solution is to edit known sample points in advance and remove samples that do not have much effect on classification. This algorithm is suitable for scenarios with large sample size.
Automatic Classification of category domains, which are smaller than the sample size. This algorithm is easy to produce incorrect scores.

From: Baidu encyclopedia-http://baike.baidu.com/view/1485833.htm

Useful: http://hi.baidu.com/hxtang/blog/item/a29e9a87b519e12dc65cc3db.html