A simple and easy-to-learn machine learning algorithm--density-based clustering algorithm Dbscan

An overview of density-based clustering algorithms recently, a density-based clustering algorithm in science, "clustering by fast search and find of density peaks" attracted attention (in my blog "The Machine Learning algorithm--the base The clustering algorithm for density peaks is also described in Chinese).    So I want to understand the density-based clustering algorithm, familiar with the density-based clustering algorithm and distance-based clustering algorithm, such as the difference between the K-means algorithm. The main goal of density-based clustering algorithm is to find high-density regions separated by low-density regions. Unlike distance-based clustering algorithms, cluster results based on distance clustering are spherical clusters, and density-based clustering algorithms can discover arbitrary-shaped clusters, which plays an important role in data with noisy points. Second, the principle of DBSCAN algorithm 1, the basic concept of DBSCAN (density-based Spatial Clustering of application with Noise) is a typical density-based clustering algorithm, In the Dbscan algorithm, the data points are divided into three categories:

• Core point. In the RADIUS EPS contains more than minpts number of points
• The boundary point. The number of points within the radius EPS is less than minpts, but falls within the neighborhood of the core point
• Noise point. A point that is neither a core point nor a boundary point

Here there are two quantities, one is the radius eps, the other is the specified number minpts. Some of the other concepts

1. EPS neighborhood. In simple terms, a collection of points that are less than or equal to the point of the EPS can be expressed as.
2. Direct density can be reached. If within the EPS neighborhood of the core object, it is said that the object from the object is directly density can be reached.
3. Density can be reached. For the object chain:, which is from the direct density of EPs and minpts, then the object is from the object about EPS and minpts density can be reached.

2, algorithm flow (flow) Three, experimental simulation in the experiment using two test data sets, the original image of the dataset is as follows: (DataSet 1) (DataSet 2) DataSet 1 is relatively simple. Obviously we can find data set 1 total two classes, DataSet 2 has four classes, below we use the Dbscan algorithm to achieve the clustering of data points: Matlab code main program [Plain]View PlainCopy

2. Percent DBSCAN
4. Clear all;
6. CLC
10. Percent Import data set
12. % data = load (' testData.txt ');
14. data = Load (' testdata_2.txt ');
18. % definition parameters EPs and minpts
20. minpts = 5;
22. Eps = Epsilon (data, minpts);
26. [M,n] = size (data);
30. x = [(1:m) ' data];
32. [M,n] = size (x);% recalculate data set size
34. types = zeros (1,m);% is used to distinguish core point 1, boundary Point 0 and noise point 1
36. dealed = zeros (m,1);% is used to determine if the point has been processed, and 0 indicates that it has not been processed
38. dis = caldistance (x (:, 2:n));
40. Number = 1;% is used to mark classes
44. Percent of each point to be processed
46. For i = 1:m
48. % found unhandled points
50. If dealed (i) = = 0
52. Xtemp = x (i,:);
54. D = Dis (i,:);% gets the distance from point I to all other points
56. IND = Find (d<=eps);% finds all points within the radius Eps
60. The type of the percent difference point
64. % Boundary point
66. If Length (Ind) > 1 && Length (Ind) < minpts+1
68. Types (i) = 0;
70. Class (i) = 0;
72. End
74. % Noise Point
76. If Length (ind) = = 1
78. Types (i) =-1;
80. Class (i) =-1;
82. dealed (i) = 1;
84. End
86. % Core point (here is the key step)
88. If Length (Ind) >= minpts+1
90. Types (xtemp) = 1;
92. Class (Ind) = number;
96. % to determine if the core point density is up to
98. While ~isempty (Ind)
100. ytemp = x (Ind (1),:);
102. Dealed (Ind (1)) = 1;
104. IND (1) = [];
106. D = Dis (ytemp),:);% find distance to IND (1)
108. Ind_1 = Find (d<=eps);
112. If Length (ind_1) >1% handles non-noise points
114. Class (Ind_1) = number;
116. If Length (ind_1) >= minpts+1
118. Types (ytemp) = 1;
120. Else
122. Types (ytemp) = 0;
124. End
128. For J=1:length (ind_1)
130. If Dealed (Ind_1 (j)) = = 0
132. Dealed (Ind_1 (j)) = 1;
134. Ind=[ind Ind_1 (j)];
136. Class (Ind_1 (j)) =number;
138. End
140. End
142. End
144. End
146. Number = number + 1;
148. End
150. End
152. End
156. % final processing of all unclassified points for noise points
158. Ind_2 = Find (class==0);
160. Class (Ind_2) =-1;
162. Types (ind_2) =-1;
166. To draw the final cluster diagram
168. On
170. For i = 1:m
172. If class (i) = =-1
174. Plot (data (i,1), data (i,2), '. R ');
176. ElseIf Class (i) = = 1
178. If types (i) = = 1
180. Plot (data (i,1), data (i,2), ' +b ');
182. Else
184. Plot (data (i,1), data (i,2), '. B ');
186. End
188. ElseIf Class (i) = = 2
190. If types (i) = = 1
192. Plot (data (i,1), data (i,2), ' +g ');
194. Else
196. Plot (data (i,1), data (i,2), '. G ');
198. End
200. ElseIf Class (i) = = 3
202. If types (i) = = 1
204. Plot (data (i,1), data (i,2), ' +c ');
206. Else
208. Plot (data (i,1), data (i,2), '. C ');
210. End
212. Else
214. If types (i) = = 1
216. Plot (data (i,1), data (i,2), ' +k ');
218. Else
220. Plot (data (i,1), data (i,2), '. K ');
222. End
224. End
226. End
228. Hold off

Distance calculation function [Plain]View PlainCopy

2. Percent calculation the distance between the midpoint and the point of the matrix
4. function [Dis] = caldistance (x)
6. [M,n] = size (x);
8. dis = zeros (m,m);
12. For i = 1:m
14. for j = i:m
16. % calculation of Euclidean distance between point I and Point J
18. TMP = 0;
20. For k = 1:n
22. TMP = tmp+ (x (I,k)-X (J,k)). ^2;
24. End
26. Dis (i,j) = sqrt (TMP);
28. Dis (j,i) = Dis (I,J);
30. End
32. End
34. End

Epsilon function [Plain]View PlainCopy

2. function [Eps]=epsilon (X,K)
6. % Function: [Eps]=epsilon (X,k)
8. %
10. % Aim:
12. % analytical to estimating neighborhood radius for DBSCAN
14. %
16. % Input:
18. % X-data matrix (m,n); M-objects, N-variables
20. % K-number of objects in a neighborhood of a object
22. % (minimal number of objects considered as a cluster)
30. [M,n]=size (x);
34. Eps= ((PROD (max (x)-min (x)) *k*gamma (. 5*n+1))/(M*sqrt (Pi.^n))). ^ (1/n);

The final result (cluster result of DataSet 1) (cluster result of DataSet 2) in the above results, the red dots represent the noise points, the points represent the boundary points, and the cross represents the core points. Different colors represent different classes. References [1] M. Ester, H. Kriegel, J. Sander, X. Xu, A density-based algorithm for discovering clusters in large spatial Databa Ses with noise, www.dbs.informatik.uni-muenchen.de/cgi-bin/papers?query=--CO
[2] M. Daszykowski, B. Walczak, D. L. Massart, looking for Natural Patterns in Data. Part 1:density Based Approach

A simple and easy-to-learn machine learning algorithm--density-based clustering algorithm Dbscan