Simple understanding of the canopy algorithm of Mahout

Canopy is an implementation of clustering algorithm

It is a fast, simple, but less accurate clustering algorithm

Canopy calculates data by two artificially defined threshold t1,t2, which can be used to classify a bunch of chaotic data into N data heaps with a certain number of rules.

Because the purpose of the canopy algorithm itself is to divide chaotic data into roughly several categories, it is less accurate

But the n categories calculated by Canopy can be used to determine the K-value in the Kmeans algorithm (because it is not possible to determine exactly how much k value is appropriate, but the Kmeans algorithm itself randomly produces the result may not be very accurate. For an explanation of the Kmeans algorithm, see Click to open the link)

The canopy algorithm flow is as follows:

(1) Determine two threshold t1,t2 (ensure that T1 must be greater than T2)

(2) Randomly select a data from the data set to calculate the distance from the data to canopy (if no canopy is currently present, the point is directly as canopy)

(3) If this distance is less than T1, then the data is marked with a weak mark, the T1 is added to the canopy (and this data can be used as a new canopy to calculate the distance of the other data to this point)

(4) If this distance is less than T2, then the data is marked with a strong mark, and the data set is deleted, at this time that the data point is close enough to the canopy, it is impossible to form a new canopy

(5) Repeat the 2-4 process until there is no data in the data set

The canopy here refers to the center of the data to be divided, with this canopy as the center, T2 as the radius, forming a small circle. The T1 is a radius that forms a great circle. The data points in the small circle range are considered to belong to this canopy and cannot be used as a new canopy to divide the data, while the data within the circle range can be used as a new canopy to divide the data.

The data after the partition is similar

The circle of the dashed line is T2, and the circle of the solid is T1

You can see that the canopy algorithm will roughly divide a bunch of cluttered data into chunks.

So the canopy algorithm is generally used in conjunction with the Kmeans algorithm to reach the user's goal.

When using the canopy algorithm, it is very important to determine the threshold value of T1,T2.

The value of the T1 is too large, resulting in more data being iterated over, resulting in excessive canopy, and too little value for the opposite effect

A T2 value is too large to cause too much data in one canopy, and vice versa.

Such a situation would result in inaccurate running results.

Simple understanding of the canopy algorithm of Mahout