Learning OPENCV--KNN algorithm

Transferred from: http://blog.csdn.net/lyflower/article/details/1728642

KNN algorithm in text classification, the idea of this method is very simple and intuitive: if a sample in the feature space in the K most similar (that is, the most adjacent in the feature space) of the sample is a category, then the sample belongs to this category. This method determines the category to which the sample is to be divided based on the category of the nearest one or several samples in the categorical decision-making.

Although the KNN method relies on the limit theorem in principle, it is only related to a very small number of adjacent samples in the class decision. Therefore, the problem of sample imbalance can be better avoided by using this method. In addition, the KNN method is more suitable than other methods, because the KNN method mainly relies on the surrounding finite sample, rather than the Discriminant class domain method to determine the category of the class, so it is more appropriate for the sample set to be divided by the cross or overlap.

The disadvantage of this method is that it is computationally large, because each text to be classified is calculated from its distance to all known samples in order to obtain its K nearest neighbor points. At present, the common solution is to pre-edit the known sample points in advance to remove the small sample of the role of classification. In addition, there is a reverse KNN method, which can reduce the computational complexity of KNN algorithm and improve the efficiency of classification.

This algorithm is suitable for the automatic classification of the class domain with large sample capacity, while those with smaller sample capacity are more prone to error points.

K Neighbor classifier has good text classification effect, the statistical analysis of simulation results shows that: as a text classifier, K nearest neighbor is second only to support vector machine, and obviously better than linear least squares fitting, naive Bayesian and neural network.

Focus:

1: Characteristic dimensionality reduction (general with Chi method)

2: Truncated algorithm (three truncation algorithms)

3: Reduce the amount of computation

Demo Code:

[CPP]View Plain copy print?

2. #include "ml.h"
4. #include "highgui.h"
6. int main (int argc, char** argv)
8. {
10. const int K = 10;
12. int I, j, K, accuracy;
14. float response;
16. int train_sample_count = 100;
18. Cvrng rng_state = cvrng (-1);//Initialize random number generator state
20. cvmat* traindata = Cvcreatemat (Train_sample_count, 2, CV_32FC1);
22. cvmat* trainclasses = Cvcreatemat (Train_sample_count, 1, CV_32FC1);
24. iplimage* img = cvcreateimage (cvsize (500, 500), 8, 3);
26. float _sample[2];
28. Cvmat sample = Cvmat (1, 2, CV_32FC1, _sample);
30. Cvzero (IMG);
34. Cvmat trainData1, TrainData2, TrainClasses1, TrainClasses2;
38. form the training samples
40. Cvgetrows (Traindata, &traindata1, 0, TRAIN_SAMPLE_COUNT/2); Returns a row of an array or a line within a certain span
42. Cvrandarr (&rng_state, &traindata1, Cv_rand_normal, Cvscalar (200,200), Cvscalar (50,50)); Fills an array with random numbers and updates the RNG state
46. Cvgetrows (Traindata, &traindata2, TRAIN_SAMPLE_COUNT/2, Train_sample_count);
48. Cvrandarr (&rng_state, &traindata2, Cv_rand_normal, Cvscalar (300,300), Cvscalar (50,50));
52. Cvgetrows (trainclasses, &trainclasses1, 0, TRAIN_SAMPLE_COUNT/2);
54. Cvset (&trainclasses1, Cvscalar (1));
58. Cvgetrows (trainclasses, &trainclasses2, TRAIN_SAMPLE_COUNT/2, Train_sample_count);
60. Cvset (&trainclasses2, Cvscalar (2));
64. Learn classifier
66. Cvknearest KNN (traindata, trainclasses, 0, False, K);
68. cvmat* nearests = Cvcreatemat (1, K, CV_32FC1);
72. for (i = 0; i < img->height; i++)
74. {
76. for (j = 0; J < img->width; J + +)
78. {
80. Sample.data.fl[0] = (float) j;
82. SAMPLE.DATA.FL[1] = (float) i;
86. Estimates the response and get the neighbors ' labels
88. Response = Knn.find_nearest (&sample,k,0,0,nearests,0);
92. Compute the number of neighbors representing the majority
94. for (k = 0, accuracy = 0; k < K; k++)
96. {
98. if (nearests->data.fl[k] = = response)
100. accuracy++;
102. }
106. Highlight the pixel depending on the accuracy (or confidence)
108. CVSET2D (IMG, I, j, response = = 1?)
110. (Accuracy > 5?) Cv_rgb (180,0,0): Cv_rgb (180,120,0)):
112. (Accuracy > 5?) Cv_rgb (0,180,0): Cv_rgb (120,120,0));
114. }
116. }
120. Display the original training samples
122. for (i = 0; i < TRAIN_SAMPLE_COUNT/2; i++)
124. {
126. Cvpoint pt;
128. Pt.x = Cvround (traindata1.data.fl[i*2]);
130. Pt.y = Cvround (traindata1.data.fl[i*2+1]);
132. Cvcircle (IMG, PT, 2, Cv_rgb (255,0,0), cv_filled);
136. Pt.x = Cvround (traindata2.data.fl[i*2]);
138. Pt.y = Cvround (traindata2.data.fl[i*2+1]);
140. Cvcircle (IMG, PT, 2, Cv_rgb (0,255,0), cv_filled);
142. }
144. Cvnamedwindow ("classifier result", 1);
146. Cvshowimage ("classifier result", IMG);
148. Cvwaitkey (0);
150. Cvreleasemat (&trainclasses);
152. Cvreleasemat (&traindata);
154. return 0;
156. }

Detailed Description: http://www.cnblogs.com/xiangshancuizhu/archive/2011/08/06/2129355.html
The improved knn:http://www.cnblogs.com/xiangshancuizhu/archive/2011/11/11/2245373.html

from:http://blog.csdn.net/yangtrees/article/details/7482890

Learning OPENCV--KNN algorithm