The concrete realization way of gossip in Cassandra

(i) the role of gossip
The Cassandra Cluster does not have a central node, and each node is in exactly the same position, and they maintain the state of the cluster through a protocol called gossip.
With gossip, each node can know which nodes are included in the cluster, and the state of those nodes, which allows any node in the Cassandra Cluster to route any key, and any node that is unavailable will have no disastrous consequences.

(ii) Introduction of the gossip agreement
The scientific name of gossip is called anti-entropy (inverse entropy). ), which is more suitable for use as synchronization information in scenarios where there is no high consistency requirement. The time the information is synchronized is probably log (n), where n indicates the number of nodes.
There are two forms of gossip: Anti-entropy and rumor-mongering.
Each node in the gossip maintains a set of states that can be represented by a key/value pair, with a version number and a large version number for the updated state.
There are 3 ways to handle messages, and the Cassandra uses a third way--PUSH-PULL-GOSSIP

(c) How gossip messages are sent
when a node starts, gets the seeds configuration in the configuration file (Cassandra.yaml) to know all the seed nodes in the cluster.
Cassandra has a gossiper that runs once every second (in the Start method of Gossiper.java) and sends a synchronization message to the other nodes according to the following rules:
1 randomly take a node that is currently alive and send it a sync request
2. Send a synchronization request to a randomly unreachable machine
3 If the node selected in the first step is not seed, or if the number of nodes currently alive is less than the seed number, send a synchronization request to a random seed
If this is not the case, consider a scenario with 4 machines, {A, B, C, D}, and they are all seed, and if they start at the same time, there may be a situation where the
1, a node up, found no living node, went to the third step, and any one seed synchronization, assuming B
2, B nodes and a to complete the synchronization, it is considered a alive , it will be synchronized with a, because a is a seed, B will no longer synchronize with other seeds
3, c node up, found that there is no living node, also go to the third step, and any one seed synchronization, assuming that the selected D
4, C nodes and D to complete the synchronization, that D alive, then it will and D synchronization, Because D is also a seed, C is no longer synchronized with other seeds
at this time formed two islands, A and B are synchronized with each other, C and D are synchronized with each other, but {a, b} and {c,d} will no longer synchronize with each other, they do not know the other side of the existence. After the second judgment is added, A and B are synchronized and only one node is found to be alive, but seed has 4, which then communicates with any seed to break the island.

(iv) Gossip data structure in Cassandra
There are 3 types of state information for gossip communication:
1, Endpointstate
2, Heartbeatstate
3, ApplicationState
The heartbeatstate is composed of generation and version, generation each boot changes to distinguish the state before and after the restart of the machine; version is only growing, incrementing before each heartbeat
The applicationstate is used to represent the state of the system, which consists of state and version, States of the node, version is incremented, and each object represents a state of the node, such as the state of the current load: (1.2, 20), Meaning that the node's load is 1.2 when the version number is 20 o'clock
Endpointstate encapsulates the mapping of a node's applicationstate (map<string, applicationstate> applicationstate_) and Heartbeatstate
The state of a node itself can only be modified by itself, and the state of the other nodes can only be updated synchronously.

(vi) What are the gossip status information
Load Information (load-information)
Migration information (migration)
Node state information (MOVE)
Boot (boot phase) node is starting
The normal node joins the token ring, which provides read
Leaving, the node is ready to leave the ring
Left, the node is kicked out of the cluster or token information is manually changed

(vii) Gossip message synchronization process

(eight) gossip message synchronization instance
(1) Node 10.0.0.1 (endpointstatemap):

Endpointstate 10.0.0.1 heartbeatstate:generation 1259909635, version 325 applicationstate "Load-information": 5.2, Generation 1259909635, Version applicationstate "bootstrapping": Bxlpassf3xd8kyks, Generation 1259909635, version A Pplicationstate "Normal": Bxlpassf3xd8kyks, Generation 1259909635, version, Endpointstate 10.0.0.2 heartbeatstate: Generation 1259911052, Version applicationstate "Load-information": 2.7, Generation 1259911052, version 2 applications Tate "bootstrapping": AUJDMFTPYUVEBTNN, Generation 1259911052, version endpointstate 10.0.0.3 heartbeatstate: Generation 1259912238, version 5 applicationstate "Load-information": 12.0, Generation 1259912238, version 3 Endpointstat E 10.0.0.4 heartbeatstate:generation 1259912942, version applicationstate "Load-information": 6.7, Generation 1259912 942, version 3 ApplicationState "normal": bj05ivc0lvrxw2xh, Generation 1259912942, version 7

(2) Node 10.0.0.2 (ENDPOINTSTATEMAP):

Endpointstate 10.0.0.1 heartbeatstate:generation 1259909635, version 324 applicationstate "Load-information": 5.2, Generation 1259909635, Version applicationstate "bootstrapping": Bxlpassf3xd8kyks, Generation 1259909635, version A Pplicationstate "Normal": Bxlpassf3xd8kyks, Generation 1259909635, version, Endpointstate 10.0.0.2 heartbeatstate: Generation 1259911052, Version applicationstate "Load-information": 2.7, Generation 1259911052, version 2 applications Tate "bootstrapping": AUJDMFTPYUVEBTNN, Generation 1259911052, version applicationstate "normal": AUJDMFTPYUVEBTNN, Generation 1259911052, version endpointstate 10.0.0.3 heartbeatstate:generation 1259812143, version 2142 applications Tate "Load-information": 16.0, Generation 1259812143, version 1803 ApplicationState "Normal": W2u1xyuc3wmppcy7, Generation 1259812143, Version 6
gossipdigestsynmessage (node 10.0.0.1):
10.0.0.1:1259909635:325
10.0.0.2:1259911052:61
10.0.0.3:1259912238:5
10.0.0.4:1259912942:18

Gossipdigestackmessage (node 10.0.0.2):
10.0.0.1:1259909635:324
10.0.0.3:1259912238:0
10.0.0.4:1259912942:0
10.0.0.2:
[ApplicationState "normal": AUJDMFTPYUVEBTNN, Generation 1259911052, version 62],
[Heartbeatstate, Generation 1259911052, version 63]

Gossipdigestack2message (node 10.0.0.1):
10.0.0.1:
Heartbeatstate:generation 1259909635, Version 325
ApplicationState "Load-information": 5.2, Generation 1259909635, version 45
ApplicationState "bootstrapping": Bxlpassf3xd8kyks, Generation 1259909635, version 56
ApplicationState "Normal": Bxlpassf3xd8kyks, Generation 1259909635, version 87
10.0.0.3:
Heartbeatstate:generation 1259912238, version 5
ApplicationState "Load-information": 12.0, Generation 1259912238, version 3
10.0.0.4:
Heartbeatstate:generation 1259912942, version 18
ApplicationState "Load-information": 6.7, Generation 1259912942, version 3
ApplicationState "Normal": bj05ivc0lvrxw2xh, Generation 1259912942, version 7