【模式識別】K-近鄰分類演算法KNN

K-近鄰（K-Nearest Neighbors, KNN）是一種很好理解的分類演算法，簡單說來就是從訓練樣本中找出K個與其最相近的樣本，然後看這K個樣本中哪個類別的樣本多，則待判定的值（或說抽樣）就屬於這個類別。

KNN演算法的步驟

• 計算已知類別資料集中每個點與當前點的距離；
• 選取與當前點距離最小的K個點；
• 統計前K個點中每個類別的樣本出現的頻率；
• 返回前K個點出現頻率最高的類別作為當前點的預測分類。

OpenCV中使用CvKNearestOpenCV中實現CvKNearest類可以實現簡單的KNN訓練和預測。

int main(){float labels[10] = {0,0,0,0,0,1,1,1,1,1};Mat labelsMat(10, 1, CV_32FC1, labels);cout<<labelsMat<<endl;float trainingData[10][2];srand(time(0)); for(int i=0;i<5;i++){trainingData[i][0] = rand()%255+1;trainingData[i][1] = rand()%255+1;trainingData[i+5][0] = rand()%255+255;trainingData[i+5][1] = rand()%255+255;}Mat trainingDataMat(10, 2, CV_32FC1, trainingData);cout<<trainingDataMat<<endl;CvKNearest knn;knn.train(trainingDataMat,labelsMat,Mat(), false, 2 );// Data for visual representationint width = 512, height = 512;Mat image = Mat::zeros(height, width, CV_8UC3);Vec3b green(0,255,0), blue (255,0,0);for (int i = 0; i < image.rows; ++i){for (int j = 0; j < image.cols; ++j){const Mat sampleMat = (Mat_<float>(1,2) << i,j);Mat response;float result = knn.find_nearest(sampleMat,1);if (result !=0){image.at<Vec3b>(j, i)  = green;}else  image.at<Vec3b>(j, i)  = blue;}}// Show the training datafor(int i=0;i<5;i++){circle(image, Point(trainingData[i][0],  trainingData[i][1]), 5, Scalar(  0,   0,   0), -1, 8);circle(image, Point(trainingData[i+5][0],  trainingData[i+5][1]), 5, Scalar(255, 255, 255), -1, 8);}imshow("KNN Simple Example", image); // show it to the userwaitKey(10000);}

使用的是之前BP神經網路中的例子，分類結果如下：
預測函數find_nearest()除了輸入sample參數外還有些其他的參數：

float CvKNearest::find_nearest(const Mat& samples, int k, Mat* results=0, const float** neighbors=0, Mat* neighborResponses=0, Mat* dist=0 )

即，samples為樣本數*特徵數的浮點矩陣；K為尋找最近點的個數；results與預測結果；neibhbors為k*樣本數的指標數組（輸入為const，實在不知為何如此設計）；neighborResponse為樣本數*k的每個樣本K個近鄰的輸出值；dist為樣本數*k的每個樣本K個近鄰的距離。另一個例子OpenCV refman也提供了一個類似的樣本，使用CvMat格式的輸入參數：

int main( int argc, char** argv ){const int K = 10;int i, j, k, accuracy;float response;int train_sample_count = 100;CvRNG rng_state = cvRNG(-1);CvMat* trainData = cvCreateMat( train_sample_count, 2, CV_32FC1 );CvMat* trainClasses = cvCreateMat( train_sample_count, 1, CV_32FC1 );IplImage* img = cvCreateImage( cvSize( 500, 500 ), 8, 3 );float _sample[2];CvMat sample = cvMat( 1, 2, CV_32FC1, _sample );cvZero( img );CvMat trainData1, trainData2, trainClasses1, trainClasses2;// form the training samplescvGetRows( trainData, &trainData1, 0, train_sample_count/2 );cvRandArr( &rng_state, &trainData1, CV_RAND_NORMAL, cvScalar(200,200), cvScalar(50,50) );cvGetRows( trainData, &trainData2, train_sample_count/2, train_sample_count );cvRandArr( &rng_state, &trainData2, CV_RAND_NORMAL, cvScalar(300,300), cvScalar(50,50) );cvGetRows( trainClasses, &trainClasses1, 0, train_sample_count/2 );cvSet( &trainClasses1, cvScalar(1) );cvGetRows( trainClasses, &trainClasses2, train_sample_count/2, train_sample_count );cvSet( &trainClasses2, cvScalar(2) );// learn classifierCvKNearest knn( trainData, trainClasses, 0, false, K );CvMat* nearests = cvCreateMat( 1, K, CV_32FC1);for( i = 0; i < img->height; i++ ){for( j = 0; j < img->width; j++ ){sample.data.fl[0] = (float)j;sample.data.fl[1] = (float)i;// estimate the response and get the neighbors’ labelsresponse = knn.find_nearest(&sample,K,0,0,nearests,0);// compute the number of neighbors representing the majorityfor( k = 0, accuracy = 0; k < K; k++ ){if( nearests->data.fl[k] == response)accuracy++;}// highlight the pixel depending on the accuracy (or confidence)cvSet2D( img, i, j, response == 1 ?(accuracy > 5 ? CV_RGB(180,0,0) : CV_RGB(180,120,0)) :(accuracy > 5 ? CV_RGB(0,180,0) : CV_RGB(120,120,0)) );}}// display the original training samplesfor( i = 0; i < train_sample_count/2; i++ ){CvPoint pt;pt.x = cvRound(trainData1.data.fl[i*2]);pt.y = cvRound(trainData1.data.fl[i*2+1]);cvCircle( img, pt, 2, CV_RGB(255,0,0), CV_FILLED );pt.x = cvRound(trainData2.data.fl[i*2]);pt.y = cvRound(trainData2.data.fl[i*2+1]);cvCircle( img, pt, 2, CV_RGB(0,255,0), CV_FILLED );}cvNamedWindow( "classifier result", 1 );cvShowImage( "classifier result", img );cvWaitKey(0);cvReleaseMat( &trainClasses );cvReleaseMat( &trainData );return 0;}

分類結果：

KNN的思想很好理解，也非常容易實現，同時分類結果較高，對異常值不敏感。但計算複雜度較高，不適於大資料的分類問題。

（轉載請註明作者和出處：http://blog.csdn.net/xiaowei_cqu 未經允許請勿用於商業用途）