Introduction to FCM clustering algorithms-reprinted

Introduction to FCM clustering algorithms

 

The FCM algorithm is a division-based clustering algorithm. Its idea is to maximize the similarity between objects divided into the same cluster, and minimize the similarity between different clusters. The fuzzy C-mean algorithm is an improvement of the ordinary C-mean algorithm. The ordinary C-mean algorithm is hard to divide data, while FCM is a Flexible Fuzzy division. Before introducing the specific FCM algorithm, we will first introduce some basic knowledge about fuzzy sets.

1. Basic knowledge of Fuzzy Sets

DescriptionMembership Function. The membership function is A function that indicates the degree to which object x belongs to set A. It is usually recorded as μ A (x ), the range of independent variables is all objects that may belong to set A (that is, all vertices in the space where set A is located). The value range is [0, 1], that is, 0 <= μ A (x) <= 1. μ A (x) = 1 indicates that x is completely affiliated with Set A, which is equivalent to the traditional concept of x in. A membership function defined on space X = {x} defines A fuzzy set A, or A fuzzy subset defined on the universe X = {x. For a limited number of objects x1, x2 ,......, Xn fuzzy set can be expressed:

(6.1)

With the concept of fuzzy set, it is not hard for an element to belong to a fuzzy set. In clustering problems, we can regard the cluster generated by clustering as a fuzzy set. Therefore, the membership of each sample point to the cluster is the value in the [0, 1] interval.

Introduction to the 2 K mean clustering algorithm (HCM, K-Means)

K-Means (K-Means), a well-known C-Means cluster, has been applied to various fields. Its core idea is as follows: the algorithm divides n vectors xj (1, 2 ..., N) divided into group c Gi (I = 1, 2 ,..., C), and find the clustering center of each group, so that the value function (or target function) of non-similarity (or distance) indicators can be minimized. When Euclidean distance is selected as the non-similarity between the vector xk and the corresponding cluster center ci in group j, the value function can be defined:

(6.2)

Here is the value function in group I. In this way, the value of Ji depends on the Gi ry and the position of ci.

Generally, if a general distance function d (xk, ci) is used to replace the vector xk in group I, the corresponding total value function can be expressed:

(6.3)

For simplicity, Euclidean distance is used as the non-similarity indicator of the vector, and the total value function is expressed as (6.2.

A divided group is generally defined by a c × n two-dimensional membership matrix U. If xj, the data point j, belongs to group I, the element uij in U is 1; otherwise, the element is 0. Once the cluster center ci is determined, the following formula (6.2) can be exported to the smallest uij:

 

(6.4)

To reiterate, If ci is the closest cluster center of xj, xj belongs to group I. Since a given data can only belong to one group, the membership matrix U has the following properties:

(6.5)

And

(6.6)

On the other hand, if uij is fixed, the optimal clustering center with the minimum formula (6.2) is the mean of all vectors in Group I:

(6.7)

Here | Gi | indicates the Gi scale or.

For batch mode operation convenience, the dataset xi (1, 2 ..., N) K-means algorithm. The algorithm repeats the following steps to determine the cluster center ci and membership matrix U:

Step 1: Initialize the cluster center ci, I = 1 ,..., C. A typical practice is to take a c point from all data points.

Step 2: Use formula (6.4) to determine the membership matrix U.

Step 3: calculate value functions based on formula (6.2. If it is less than a fixed threshold value, or its change relative to the last value function quality is less than a threshold value, the algorithm stops.

Step 4: Modify the clustering center according to formula (6.5. Return to step 2.

The algorithm is iterative and cannot converge to the optimal solution. The performance of the K-means algorithm depends on the initial position of the cluster center. Therefore, to make it desirable, either use some front-end methods to obtain the initial clustering center, or use different initial clustering centers each time to run the algorithm multiple times. In addition, the above algorithm is just a representative method. We can initialize an arbitrary membership matrix before executing the iteration process.

The K-means algorithm can also be run online. At this time, the corresponding cluster center and corresponding group are exported through the time average. That is, for the given data point x, the algorithm calculates the closest cluster center ci and uses the following formula to correct it:

(6.8)

This online formula is essentially embedded with the learning rules of many unsupervised learning neural networks.

3. fuzzy C-means clustering

Fuzzy CMean Clustering(FCM), known as fuzzyISODATAIs a clustering algorithm that uses the degree of membership to determine the degree to which each data point belongs to a certain cluster. Bezdek proposed this algorithm in 1973 as an improvement in the early hard C-means clustering (HCM) method.

FCM sets n vectors xi (I = 1, 2 ,..., N) it is divided into c fuzzy groups, and the clustering center of each group is obtained to minimize the value functions of non-similarity indicators. The main difference between FCM and HCM is that FCM uses fuzzy classification to determine the degree to which each given data point belongs to each group based on the degree of membership between 0 and 1. In line with the introduction of fuzzy division, the membership matrix U allows elements with values between 0 and 1. However, in addition to normalization, the membership of a dataset is equal to and total 1:

(6.9)

Then, the value function (or target function) of FCM is a general form of formula (6.2:

(6.10)

Uij is between 0 and 1. ci is the cluster center of Fuzzy Group I. dij = | ci-xj | it is the Euclidean distance between the I-th cluster center and the j-th data point; it is also a weighted index.

Construct the following new target function to obtain the necessary conditions for the (6.10) formula to reach the minimum value:

(6.11)

Here, lj, j = 1 to n, which is the Laplace multiplier of n constraints in the (6.9) formula. Evaluate the derivation of all input parameters so that formula (6.10) reaches the minimum required condition:

(6.12)

And

(6.13)

The fuzzy C-means clustering algorithm is a simple iteration process based on the preceding two necessary conditions. During batch processing, FCM uses the following steps to determine the cluster center ci and membership matrix U [1]:

Step 1: Use a random number between 0 and 1 to initialize the membership matrix U to satisfy the constraints in formula (6.9 ).

Step 2: Use formula (6.12) to calculate the c cluster center ci, I = 1 ,..., C.

Step 3: calculate value functions based on formula (6.10. If it is less than a fixed threshold value, or its change relative to the previous value function value is less than a threshold value, the algorithm stops.

Step 4: Calculate the new U matrix using (6.13. Return to step 2.

The preceding algorithm can also initialize the cluster center and then execute the iteration process. It cannot ensure that FCM converges to an optimal solution. The algorithm performance depends on the Initial Clustering center. Therefore, we either use another rapid algorithm to determine the initial cluster center, or start the algorithm with different initial cluster centers each time, and run FCM multiple times.

4. Application of FCM Algorithm

From the above discussion, we can easily see that the FCM algorithm requires two parameters: one is the number of clusters C, and the other is the parameter m. Generally, C is much smaller than the total number of cluster samples, and C> 1 must be ensured. For m, it is a flexible parameter for controlling algorithms. If m is too large, the clustering effect will be very good, and if m is too small, the algorithm will be close to the HCM clustering algorithm.

The output of the algorithm is a fuzzy matrix of the C cluster center vector and C * n, which indicates that each sample point belongs to each class. Based on this matrix, the maximum membership principle in the fuzzy set can be used to determine which class each sample point belongs. The clustering center represents the average features of each class and can be considered as the representative point of this class.

From the algorithm derivation process, it is not difficult to see that the algorithm will have a good effect on clustering data that satisfies the normal distribution. In addition, the algorithm is sensitive to isolated points.