Python/matlab Realization KNN algorithm

Source: Internet
Author: User

PythonAlgorithm pseudo code: For each point in the dataset of the Unknown category attribute, do the following: 1) calculates the distance between the point in the data set of the known category and the current point, and 2) sorts by the increment order of distance, 3) selects the K point with the minimum distance from the current point; 4) determines the frequency of occurrence of the category ; 5) returns the category with the highest frequency of the first K points as the predicted classification of the current point.   Euclidean distance calculation: (1) Euclidean distance between two point Xa (x1,y1) and XB (X2,y2) on two-dimensional plane: (2) Euclidean distance between two point Xa (x1,y1,z1) and XB (X2,Y2,Z2) in three-dimensional space: (3) Two n-dimensional vector xa (x11,x12, The Euclidean distance:  algorithm between x1n and XB (x21,x22,..., x2n) is implemented as follows: 1, Evaluation indicator DEF createdataset ():    Group = Array ([[1.0,1.1],[ 1.0,1.0],[0,0],[0,0.1]])     labels = [' A ', ' a ', ' B ', ' B ']    return group, LABELS2, KNN core algorithm #inx  & nbsp Data to be detected #dataset  dataset #labels    result set #k      length to compare def classify0 (InX, DataSet, labels, k): & nbsp   Datasetsize = dataset.shape[0]                #计算有多少行     # tile (i NX, (datasetsize,1)) generates a matrix corresponding to the InX dimension, making it easy to do poor     Diffmat = Tile (InX, (datasetsize,1))-dataset    Sqdiffmat = diffmat**2                          #差求平方     Sqdistances = Sqdiffmat.sum (Axis=1) &nbsp           # axis=0, which represents the column Axis=1, represents the row.     distances = sqdistances**0.5                  #开方     Sorteddistindicies = Distances.argsort ()       #argsort () sorting, subscript     classcount={}    For I in range (k):        Voteilabel = labels[sorteddistindicies[i]]  #通过下标索引分类     &NBS P   # By constructing a dictionary, record classification frequency         Classcount[voteilabel] = Classcount.get (voteilabel,0) + 1    & nbsp   # Sort fields by value (from large to small)         Sortedclasscount = sorted (Classcount.items (), Key=lambda ClassCount: CLASSCOUNT[1], reverse=true)     return sortedclasscount[0][0] 3, test class # test if __name__ = = "__main__":    DataSet = Array ([[1.0, 1.1], [1.0, 1.0], [0.0, 0.0], [0.0, 0.1]])     labels = [' A ', ' a ', ' B ', ' B ']    X = [1.2, 1.1]    Y = [0.1, 0.1]    k = 3   LabelX =  classify0 (x,dataset,labels,k)     labely =  classify0 (y,dataset,labels,k)     Print ("Your input is:", X, "and classified to class:", LabelX)     print ("Your input is:", Y, "and classified To class: ", labely) Results: Your input is: [1.2, 1.1] and classified to class:  ayour input is: [0.1, 0.1] and classified To class:  B4, notethe KNN algorithm is expensive because it calculates the distance from each sample point to all other points.the distance of KNN algorithm is usually selected according to the actual sample point.the distance threshold of KNN algorithm is selected according to the degree of dispersion concentration of the sample. Experience generally selects the mean variance of the sample point set.matlab5, willful matlab right, MATLAB is so wayward only need a row, the other is read Data  train_data=load (' Sample_feature.txt '); Train_label=load (' Train_ Label.txt '); Test_data=load (' features.txt '); K=knnclassify (test_data,train_data,train_label,3, ' cosine ', ' random ')  train_data Preservation is the training sample characteristics, the requirements are most representative of this category, not necessarily more, of course, not too little; Train_label Save is the sample label, such as 0,1,2 and so on, casually set, only can differentiate on the line, the specific format can be: Test_ The data test file holds the characteristics of the test;  key functions: Knnclassify is the classifier;  function call form using nearest neighbor: 1. CLASS = Knnclassify (sample,training,group)   label and training data must have the same number of rows; The training data and the test data must have the same columns; the function will either lose values or ignore the line for invalid values or null values. 2.CLASS = Knnclassify (sample,training,group,k) This function allows you to set the distance matrix form, such as:              ' Euclidean '     Euclidean distance, default               ' cityblock '     absolute difference and   &NBS P           ' cosine '      angle range               ' Corre Lation '   related distance               ' Hamming '       Hamming distance 3. CLASS =knnclassify (sample,training, Group,k,distance,rule) This function allows you to choose how to categorize the sample, as you can choose:                ' nearest '   Most recent K's most               ' random '     random most         &N Bsp     ' consensus '   voting, default   and I choose the function form: k=knnclassify (test_data,train_data,train_label,3, ' cosine ' , ' random '); 

Python/matlab implementation KNN algorithm

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.