Implementation of gossip in cassandra

(1) Role of Gossip
Cassandra clusters have no central nodes and each node has the same status. They maintain the cluster status through a protocol called gossip.
Through gossip, each node can know which nodes are included in the cluster and their statuses, which enables any node in the Cassandra cluster to route any key, unavailability of any node will not cause disastrous consequences.

(2) Introduction to the Gossip Protocol
The name of gossip is Anti-entropy (inverse entropy ?), It is suitable for synchronous information in scenarios without high consistency requirements. The synchronization time is about log (N), where N indicates the number of nodes.
Gossip has two forms: anti-entropy and rumor-mongering.
Each node in gossip maintains a group of statuses, which can be represented by a key/value pair and a version number. The updated status is the version number.
There are three ways to process messages. Cassandra adopts the third method -- Push-pull-gossip

(3) how to send a Gossip message
When a node is started, obtain the seeds configuration in the configuration file (cassandra. yaml) to know all the seed nodes in the cluster.
Cassandra has a Gossiper that runs every second (in the start method of Gossiper. java) and sends a synchronous message to other nodes according to the following rules:
1. Randomly fetch a living node and send a synchronous request to it
2. Send a synchronization request to a random machine that cannot be reached
3. If the Node Selected in step 1 is not seed or the number of currently active nodes is less than seed, a synchronous request is sent to any seed.
If you do not have this judgment, consider this scenario. There are four machines, {A, B, C, D}, and they are all set to seed. If they are started at the same time, this may happen:
1. When node A is started and no living node is found, it goes to Step 3 and synchronizes with any seed. Assume that node B is selected.
2. If node B and node A are synchronized, A is considered to be alive and will be synchronized with node A. Because node A is A seed, B will not be synchronized with other seed.
3. When node C is started and no living node is found, it also goes to Step 3 and synchronizes with any seed. Assume that the node D is selected this time.
4. C node and D are synchronized. If D is considered to be alive, it will be synchronized with D. Because D is also a seed, C will not be synchronized with other seed.
At this time, two isolated islands are formed. A and B are synchronized with each other. C and D are synchronized with each other, but {A, B} and {C, D} are not synchronized with each other, they do not know each other's existence.
After the second judgment is added, A and B are synchronized, and only one node is found to be alive, but there are four seed nodes. Then, they will communicate with any other seed to break this isolated island.

(4) Gossip Data Structure in Cassandra
There are three main statuses of gossip communication:
1. EndPointState
2. HeartBeatState
3. ApplicationState
HeartBeatState is composed of generation and version. generation changes every time it is started. It is used to differentiate the status before and after the machine restarts. Version can only increase, but increases before each heartbeat.
ApplicationState indicates the state of the system, which is composed of state and version. state indicates the state of the node. version is incremental. Each object indicates a state of the node, for example, the current load status is like this: (1.2, 20), meaning that when the version number is 20, the load of this node is 1.2
EndPointState encapsulates the appling (Map <String, ApplicationState> ApplicationState _) and HeartBeatState of a node.
The status of a node can only be changed by itself. The status of other nodes can only be updated synchronously.

(6) What are the Gossip status information?
LOAD-INFORMATION)
MIGRATION Information (MIGRATION)
Node status information (MOVE)
The BOOT node is starting.
The NORMAL (NORMAL) node is added with a Token ring to provide read
LEAVING: the node is about to leave the Ring.
LEFT: the node is kicked out of the cluster or the Token information is manually changed.

(7) Gossip message synchronization process

(8) Gossip message synchronization instance
(1) Node 10.0.0.1 (endPointStateMap ):

 Endpointstate 10.0.0.1 <br/> heartbeatstate: Generation 1259909635, version 325 <br/> applicationstate "load-Information": 5.2, generation 1259909635, version 45 <br/> applicationstate "bootstrapping ": bxlpassf3xd8kyks, generation 1259909635, version 56 <br/> applicationstate "normal": bxlpassf3xd8kyks, generation 1259909635, version 87 <br/> endpointstate 10.0.0.2 <br/> heartbeatstate: Generation 1259911052, version 61 <br/> applicationstate "load-Information": 2.7, generation 1259911052, version 2 <br/> applicationstate "bootstrapping": aujdmftpyuvebtnn, generation 1259911052, version 31 <br/> endpointstate 10.0.0.3 <br/> heartbeatstate: Generation 1259912238, version 5 <br/> applicationstate "load-Information": 12.0, generation 1259912238, version 3 <br/> endpointstate 10.0.0.4 <br/> heartbeatstate: Generation 1259912942, Version 18 <br/> applicationstate "load-Information": 6.7, generation 1259912942, version 3 <br/> applicationstate "normal": bj05ivc0lvrxw2xh, generation 1259912942, version 7

(2) Node 10.0.0.2 (endPointStateMap ):

EndPointState 10.0.0.1 <br/> HeartBeatState: generation 1259909635, version 324 <br/> ApplicationState "load-information": 5.2, generation 1259909635, version 45 <br/> ApplicationState "bootstrapping ": bxLpassF3XD8Kyks, generation 1259909635, version 56 <br/> ApplicationState "normal": bxLpassF3XD8Kyks, generation 1259909635, version 87 <br/> EndPointState 10.0.0.2 <br/> HeartBeatState: generation 1259911052, version 63 <br/> ApplicationState "load-information": 2.7, generation 1259911052, version 2 <br/> ApplicationState "bootstrapping": AujDMftpyUvebtnn, generation 1259911052, version 31 <br/> ApplicationState "normal": AujDMftpyUvebtnn, generation 1259911052, version 62 <br/> EndPointState 10.0.0.3 <br/> HeartBeatState: generation 1259812143, version 2142 <br/> ApplicationState "load-information": 16.0, generation 1259812143, version 1803 <br/> ApplicationState "normal": W2U1XYUC3wMppcY7, generation 1259812143, version 6
GossipDigestSynMessage (node 10.0.0.1 ):
10.0.0.1: 1259909635: 325
10.0.0.2: 1259911052: 61
10.0.0.3: 1259912238: 5
10.0.0.4: 1259912942: 18

GossipDigestAckMessage (node 10.0.0.2 ):
10.0.0.1: 1259909635: 324
10.0.0.3: 1259912238: 0
10.0.0.4: 1259912942: 0
10.0.0.2:
[ApplicationState "normal": AujDMftpyUvebtnn, generation 1259911052, version 62],
[HeartBeatState, generation1259911052, version 63]

GossipDigestAck2Message (node 10.0.0.1 ):
10.0.0.1:
HeartBeatState: generation 1259909635, version 325
ApplicationState "load-information": 5.2, generation 1259909635, version 45
ApplicationState "bootstrapping": bxLpassF3XD8Kyks, generation 1259909635, version 56
ApplicationState "normal": bxLpassF3XD8Kyks, generation 1259909635, version 87
10.0.0.3:
HeartBeatState: generation 1259912238, version 5
ApplicationState "load-information": 12.0, generation 1259912238, version 3
10.0.0.4:
HeartBeatState: generation 1259912942, version 18
ApplicationState "load-information": 6.7, generation 1259912942, version 3
ApplicationState "normal": bj05IVc0lvRXw2xH, generation 1259912942, version 7