[Go] Community Discovery (Community Detection) algorithm

Author: peghoty

Source: http://blog.csdn.net/peghoty/article/details/9286905

Community Discovery (Community Detection) algorithm is used to discover the community structure in the network, and it can be regarded as a clustering algorithm .

Here is a PPT report for me to share with you.

A community is a sub-graph that contains vertices and edges.

Below we take the Sina Weibo user corresponding Network diagram as an example, to introduce the corresponding Community discovery algorithm .

There is a connection between the users of mutual concern, mainly to simplify the model, at this time the corresponding graph is an no-map.

Of course, we can also use one-way attention to build the edge, this time will be to the graph.

This definition looks very awkward, but with layers of derivation, the following mathematical expressions (4.2) can be obtained. The random network in the definition is also known as the Null Model, and is constructed by:

The null model used have so far been a random graph with the same number of nodes, the same number of edges and the same de Gree distribution as in the original graph, but with links among nodes randomly placed.

Note that (4.2) is for the even if graph, so the m here represents the number of non-aligned edges, and the node I and node J have edges connected, then the nodes (I, j) contribute only one edge to M .

The label Propagation algorithm (LPA) is relatively straightforward:

The first step: Assign a unique label to all nodes;

The second step is to refresh all the nodes ' labels on a round-by-wheel basis until the convergence requirement is reached. For each round of refreshes, the Rules for node label refreshes are as follows:

For a node, the label of all its neighbor nodes is examined and counted to assign the label that has the most number of occurrences to the current node. When the label with the most number is not unique, choose one randomly.

Note: The notation in the algorithm n_n^k represents the set of all nodes in the neighbor of node N that are labeled K.

Slpa introduces the concept of two comparative images of Listener and Speaker , which you can understand: in the process of refreshing a node tag, any node that is selected as a Listener, all its neighbor nodes Is its speaker, speaker usually more than one, a large group of speaker in rushes, listener in the end who should listen to it? Then we need to make a rule.

In LPA, we make decisions with the most frequently occurring labels, which is actually a rule. Only in the SLPA framework, the rules are selected more (can be specified by the user).

Of course, compared to LPA, the biggest feature of SLPA is that it records the sequence of historical labels for each node during the refresh iteration (such as iteration T, each node holds a sequence of length T, as shown), and when the iteration is stopped, Statistics on the frequency of each (XOR) tag appearing in each node's history tag sequence are filtered out by a given threshold, which is the label of the node (usually multiple).

Slpa was later renamed as Ganxis, and the package is still being updated ...

Here is a brief description of the above diagram: the node with question marks is the node of the label to be determined, black solid point for its neighbor nodes, their labels are known, note that the label is a sequence of two-dollar pairs, the first component of each element of the sequence represents its label, The second component represents the probability that the node belongs to the community of that label (or probability, called belonging coefficent), so the sum of its probabilities for each node equals 1.

We follow these steps to determine the label with the question mark node:

1. Get the list of all the distinct tags in the neighbor node and accumulate the corresponding belonging coefficent values.

2. Belonging the list of coefficent values, dividing the belonging coefficent value of each label in the list by C1 (C1 is the maximum value of belonging coefficent in the list).

3. Filter. If the normalized belonging Coefficent value (already between 0 and 1) in the list is less than a certain threshold p (parameters specified beforehand), the corresponding two-tuple is removed from the list.

4. Once again, do the normalization. Since filtering, the sum of each belonging Coefficent value in the remaining list does not necessarily equal 1, so each belonging coefficent value needs to be divided by C2 (C2 represents the sum of belonging coefficent values).

After the four steps above, the labels in the list are identified as labels with a question mark node.

Here, we make a brief introduction to the Fast unfolding algorithm, which is divided into the following two phases:

The first stage: assign each node to a unique community first, and then move the nodes sequentially between those communities. How do you move it? In the case of node I, it has three neighbor nodes J1, J2, J3, we try to move node I to J1, J2, J3 in the community, and calculate the corresponding modularity change value, which change the maximum of the node I moved to the corresponding community (of course, here we ask for the largest The modularity change value to be positive, if the change value is negative, then the node I will remain fixed. This method iterates over and over until any node in the network is moved without improving the total modularity value.

Second stage: The first stage of the community as a new "node" (a community corresponding to one), the re-construction of the sub-map, the weight of the two new "nodes" between the weights of the corresponding two communities the sum of the weights of each side.

We call these two phases together as a pass, and it's clear that the pass can go on.

As we can see from the above description, this algorithm contains a hierarchy structure, just like the aggregation of all junior high school students, first we can aggregate them according to the class, and further on this basis according to the grade to aggregate, two times aggregation can be seen as a community discovery results, It depends on what level and level you want to converge.

The DCLP algorithm is a variant of LPA, which introduces a parameter to limit the spread of each label, which effectively controls the generation of Monster (very large community, much larger than other community).

Finally, we give some experimental results .

Comparing the two tables above, it is shown that the top 5 communities of the SDCLP algorithm are more homogeneous.

[Go] Community Discovery (Community Detection) algorithm