KNN algorithm example for machine learning using python only, and pythonknn algorithm example

Source: Internet
Author: User

KNN algorithm example for machine learning using python only, and pythonknn algorithm example

The previous article briefly introduced linear regression, logistic regression, Bayesian classification, and simple implementation using python. This article introduces the simpler knn and k-Nearest Neighbor algorithms (kNN, k-NearestNeighbor ).

K-Nearest Neighbor (kNN, k-NearestNeighbor) is one of the simplest machine learning classification algorithms, the core idea is to use k samples closest to the target to represent the target classification (these k samples are the most similar to the target data ).

Principle

The core idea of kNN algorithm is to use k sample data closest to each other (multiple Distance Measurement Methods) to represent the classification of target data.

Specifically, there is a training sample set. Each sample contains data features and classification values.

Enter new data, compare the data with each sample in the training sample set, and find the nearest k. Among the k data sets, the category with multiple occurrences is displayed, it can be used as a classification of new data.

For example:

Determine the shape of green. If k is 3, it is a triangle. If k is equal to 5, it is a square.

Therefore, this method has the following features:

  1. Supervised Learning: training samples contain classification information.
  2. Simple Algorithm, easy to understand and implement
  3. The result is affected by the k value. k generally does not exceed 20.
  4. A large amount of computing requires calculating the distance from each sample in the sample set.
  5. Inaccurate results due to unbalanced training sample set

Next we will use oython to make a simple implementation and try to use it for dating websites.

Python

Def classify (classification, dataSet, labels, k): "defines knn algorithm classifier function: param classification: Test Data: param dataSet: training data: param labels: Category: param k: k value: return: Category "dataSetSize = dataSet. shape [0] # shape (m, n) m column n features diffMat = np. tile (partition, (dataSetSize, 1)-dataSet sqDiffMat = diffMat ** 2 sqDistances = sqDiffMat. sum (axis = 1) distances = sqDistances ** 0.5 # Euclidean distance from sortedDistIndicies = distances. argsort () # sort and Return index classCount ={} for I in range (k): voteIlabel = labels [sortedDistIndicies [I] classCount [voteIlabel] = classCount. get (voteIlabel, 0) + 1 # default 0 sortedClassCount = sorted (classCount. items (), key = lambda d: d [1], reverse = True) return sortedClassCount [0] [0]

The steps of the algorithm are described in detail. The above calculation is matrix calculation, and the following function is an algebraic operation.

Def classify_two (dataSet, dataSet, labels, k): m, n = dataSet. shape # shape (m, n) m column n features # Calculate the Euclidean distance from test data to each point distances = [] for I in range (m ): sum = 0 for j in range (n): sum + = (sums [j]-dataSet [I] [j]) ** 2 distances. append (sum ** 0.5) sortDist = sorted (distances) # category of k recent values classCount ={} for I in range (k): voteLabel = labels [distances. index (sortDist [I])] classCount [voteLabel] = classCount. get (voteLabel, 0) + 1 #0: map default sortedClass = sorted (classCount. items (), key = lambda d: d [1], reverse = True) return sortedClass [0] [0]

With the classifier above, the simplest experiment is as follows:

def createDataSet():  group = np.array([[1, 1.1], [1, 1], [0, 0], [0, 0.1]])  labels = ['A', 'A', 'B', 'B']  return group, labels

The above is a simple training sample set.

if __name__ == '__main__':  dataSet, labels = createDataSet()  r = classify_two([0, 0.2], dataSet, labels, 3)  print(r)

Execute the above function: we can see that output B, [0, 0.2] should be classified into B class.

The preceding is the simplest kNN classifier. The following is an example.

KNN is used to determine the popularity of people on dating websites

Some data in the training sample set is as follows:

40920 8.326976 0.953952 314488 7.153469 1.673904 226052 1.441871 0.805124 175136 13.147394 0.428964 138344 1.669788 0.134296 1

The first column indicates the number of frequent flyer miles per year, the second column indicates the percentage of time consumed by video games, and the third column indicates the number of liters of ice cream consumed per week. The fourth column indicates the classification result. The values 1, 2, and 3 are not liked. They are attractive and attractive.

Convert data to numpy.

# Convert text to numpydef file2matrix (filepath = "datingSet.csv"): dataSet = np. loadtxt (filepath) returnMat = dataSet [:, 0:-1] classlabelVector = dataSet [:,-1:] return returnMat, classlabelVector

First, we have a sense of data, know which features affect classification, and perform visual data analysis.

#2, 3 columns of data for analysis def show_2_3_fig (): data, cls = file2matrix () fig = plt. figure () ax = fig. add_subplot (111) ax. scatter (data [:, 1], data [:, 2], c = cls) plt. xlabel ("playing game") plt. ylabel ("Icm Cream") plt. show ()

As you can see, there is no obvious classification.

We can see that different people have obvious differences based on features. Therefore, kNN can be used for classification and prediction.

Since distance is used later, the data has a greater impact, for example, the gap between flight mileage and ice cream quantity is too large. Therefore, data must be normalized.

# Data normalization def autoNorm (dataSet): minVal = dataSet. min (0) maxVal = dataSet. max (0) ranges = maxVal-minVal normDataSet = np. zeros (dataSet. shape) m, n = dataSet. shape # Row, feature normDataSet = dataSet-minVal normDataSet = normDataSet/ranges return normDataSet, ranges, minVal

Measure the accuracy of an algorithm.

Knn algorithms can be measured by accuracy or error rate. The error rate is 0, indicating that the classification is good.

Therefore, 10% of the training samples can be used for testing and 90% for training.

# Define the test algorithm function def datingClassTest (h = 0.1): hoRatio = h datingDataMat, datingLabels = file2matrix () normMat, ranges, minVals = autoNorm (datingDataMat) m, n = normMat. shape numTestVecs = int (m * hoRatio) # Number of test data rows errorCount = 0 # Number of error categories # use the first 10% of the data to test for I in range (numTestVecs ): classifierResult = classify (normMat [I,:], normMat [numTestVecs: m,:], datingLabels [numTestVecs: m], 3) # print ('the classifier cam E back with: % d, the real answer is: % d' % (int (classifierResult), int (datingLabels [I]) if classifierResult! = DatingLabels [I]: errorCount + = 1 print ("the total error rate is: % f" % (errorCount/float (numTestVecs )))

Adjust the testing ratio and compare the results.

Use knn for prediction.

With the training sample and classifier, new data can be predicted. Simulate the data and make predictions as follows:

# Simple Prediction def classifypersion (): resultList = ["none", 'Not at all', 'in small docs ', 'In large docs'] # simulate data ffmiles = 15360 playing_game = 8.545204 ice_name = 1.340429 datingDataMat, datingLabels = file2matrix () normMat, ranges, minVals = autoNorm (datingDataMat) inArr = np. array ([ffmiles, playing_game, ice_name]) # predicted Data normalization inArr = (inArr-minVals)/ranges classifierResult = classify (inArr, normMat, datingLabels, 3) print (resultList [int (classifierResult)])

You can see the basic classification.

For complete code and data, see:

Github: kNN

Summary

  1. KNN
  2. Supervised Learning
  3. Data Visualization
  4. Data normalization without affecting computing

The above is all the content of this article. I hope it will be helpful for your learning and support for helping customers.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.