Machine learning K Nearest neighbor algorithm--1, KNN classification algorithm (basic principle) __ algorithm

operating Environment (WIN7):

1. Download Python3.3.exe

2. Download the Numpy-1.9.1-win32-superpack-python3.3.exe in http://sourceforge.net/projects/numpy/files/

the basic principle of KNN classification algorithm:

Given training set A and test sample T, select the K training samples closest to T and a, and select the most frequently occurring labels in these training samples as new labels for the test sample T.

the pseudo-code flow of the KNN classification algorithm:

1. Prepare non-tagged test data: inx=[0,0]

2. Prepare sample data (eigenvalues): dataset={[[1,1],[1,2],[0,0],[0,1]])

3. Prepare sample data (tag/target value): labels=[' A ', ' a ', ' B ', ' B ']

4. Calculate the distance between each point in the known sample data and the current test point

4.1, expand the test data with NumPy expansion array function tile, according to the number of rows of sample data to expand to:

Diffmat=tile (InX, (4,1)) ={[[0,0],[0,0],[0,0],[0,0]}

The simple usage of the tile function is as follows:

4.2, using the Euclidean distance formula (below), calculates the distance between the test vector point and the sample vector point:

Diffmat=tile (InX, (4,1))-dataset #相减

Sqdiffmat=diffmat**2 #平方

Sqdistances=sqdiffmat.sum (Axis=1) #axis =1 represents the addition of the row direction, which is the sum of squares

distances=sqdistances**0.5 #开方

5, the value of the distance in ascending order from small to large

6, the minimum distance of the K points of the classification value

7. Calculate the most frequently occurring labels in k points

The code for the KNN classification algorithm:

The test results are as follows: