K-means Clustering algorithm

Clustering

Clustering is the main content is to classify the sample, the same class of samples put together, all samples will eventually form K clusters, it belongs to unsupervised learning.

Core Ideas

According to the given K value and K initial centroid, each point in the sample is divided into the nearest class cluster, and when all points are allocated, the centroid is recalculated based on all points of each class cluster, usually by means of the average, and then each point is divided into the new cluster of the nearest distance, and the operation is continuously recycled. Until the centroid no longer changes or reaches a certain number of iterations. Mathematically it can be proved that the K-means is convergent.

Pseudo code

随机选择k个初始质心while(trueif(质心与上一次质心一样or达到最大迭代次数)   break;}

Disadvantages

1. The number of class clusters needs to be determined beforehand.
2. The selection of centroid affects the final clustering results.

Code implementation

 fromNumPyImport*ImportMatplotlib.pyplot asPlt fromSklearn.clusterImportKmeans def Kmeans(DataSet, k):Samplenum, col = dataset.shape cluster = Mat (Zeros (Samplenum,2)) Centroids = Zeros ((k, col))# #choose Centroids     forIinchRange (k): index = int (Random.uniform (0, Samplenum)) centroids[i,:] = Dataset[index,:] clusterchanged =True     whileClusterchanged:clusterchanged =False         forIinchRange (samplenum): mindist = sqrt (sum (Power (centroids[0,:]-dataset[i,:],2)) Minindex =0             forJinchRange1, k): distance = sqrt (sum (Power (CENTROIDS[J,:]-dataset[i,:),2)))ifDistance < Mindist:mindist = Distance Minindex = JifCluster[i,0]! = Minindex:clusterchanged =TrueCluster[i,:] = Minindex, mindist**2         forJinchRange (k): Pointsincluster = Dataset[nonzero (cluster[:,0]. A = = j) [0]] centroids[j,:] = mean (pointsincluster, Axis =0)returnCentroids, Clusterdataset = [[1,1],[3,1],[1,4],[2,5],[ One, A],[ -, One],[ -, A],[ One, -],[ -, A],[ -,Ten],[ -, the],[ -, -],[ -, One],[ in, the]]dataset = Mat (dataSet) k =3Centroids, cluster = Kmeans (DataSet, k) samplenum, col = Dataset.shapemark = [' or ',' OB ',' og '] forIinchRange (samplenum): markindex = Int (cluster[i,0]) Plt.plot (Dataset[i,0], Dataset[i,1], Mark[markindex]) mark = [' +r ',' +b ',' +g '] forIinchRange (k): Plt.plot (Centroids[i,0], Centroids[i,1], Mark[i], markersize= A) Plt.show ()

Results:

More convenient to use machine learning library directly

[[1,1],[3,1],[1,4],[2,5],[11,12],[14,11],[13,12],[11,16],[17,12],[28,10],[26,15],[27,13],[28,11],[29,15]]3markers = [‘^‘‘o‘‘x‘]cls =KMeans(k).fit(dataSet)forin range(k):    members=cls.labels_==i    plt.scatter(dataSet[members,0],dataSet[members,1],marker=markers[i])plt.show()

Welcome attention to the public number:

K-means Clustering algorithm