Cassandra Source Code Summary

1. Org.apache.cassandra.dht Package

A. Interface ringposition

Public Token GetToken ();

Public boolean isminimum (Ipartitioner partitioner);

This class is a token position for ring rings, where two methods GetToken () return token values, and another method Isminimum () is used to determine whether token is the smallest token value, and this interface is often inherited

B. Class Range

This class describes the interval of the hash ring, including the left endpoint and the right endpoint, the class encapsulates a lot of APIs about the interval of the loop, including whether the interval is intersected, whether the interval is included, the interval is set, the interval is different, etc., which is mainly used for some operations on the interval of the hash ring. As for when these APIs are invoked, depending on the actual situation, it depends on the role of the API.

Public Boolean contains (range<t>);//To determine whether two Range is equal;

Public Boolean intersects (range<t> that);//Determine whether Range intersects

Public boolean equals (Object o);//Judge Equality

C. Abstract class token inheritance Ringposition

This class is a Token,token value on the hash ring that determines the position of the node (the nodes are arranged in the order of the token according to their values, Also determines where the copy is placed (the first copy of the replica is placed on the first node that is larger than his token value by the partition Partitioner, and the rest is placed sequentially in accordance with the copy placement policy)

The concept of token is very important, token is also the only distinction between replicas and nodes, token class calls many. Because the token itself is also an abstract class, it is inherited by many classes and inherited by any class, so the token value is what type. If inherited by Longtoken, then the token value is long and if Stringtoken inherits, then token is string type

Compare () comparison function, Equals () to determine whether token is equal, hashcode () generate token hash value;

Tokenfactory is a class of key values and token that have many methods of encapsulation.

Longtoken inherits token because the token is generic, so the type long replaces T, then the token value is long, and so is the other reason.

Ipartitoner is the interface of the partitioning device, and then the abstract class Abstractpartitioner inherits Iparttioner,randompartitioner, Murmur3partitioner, Localpartitioner inherits the abstract class Abstractpartitioner,ipartitioner encapsulates the API for token, has the midpoint () function to get the middle token function, and gets the smallest token, and token generation function GetToken (Bytebuffer key), this is the most important method, which is the token generation algorithm, Cassandra the default partition is Murmur3partioner, The hash algorithm for generating token is Md5hash value, murmurhash.hash3_x64_128 ();

Org.apache.cassandra.gms

 

This package is mainly about the communication process between the Cassandra Gossip protocol nodes to achieve data consistency. The main process is that a node sends a message to any online node (if it does B), Gossipdigestsyn (The SYN message encapsulates the information of the other nodes stored in Node A), Endpointstate is the information that encapsulates the node, class contains the heartbeatstate heartbeat information, as well as the application of data information applicationstate and so on, B received a sent SYN information, first update the local heartbeat queue, when satisfied with the remote generation> Local generation or, when the remote generation= local generation&& maxversion> local maxversion, updates the local heartbeat queue. Also, based on the comparison summary data and local data, generate the summary data and state data to be returned, and then encapsulate the gossipdigestack message to the A,a to receive the message, first update its own heartbeat queue, then update the local state data, generate the state data to be returned, and encapsulate into GossipDigestACK2 the message, and then sends it to B,B to receive a message from a, update its own heartbeat queue and update its own local data. The bottom drawing explains how the work works.

 

 

The Code section explains:

The Endpointstate class includes objects from the Heartbeatstate and applicationstate two classes, storing information about the heartbeat and the application data,

      public void Run () {try {//wait on messaging service to start Liste

                Ning Messagingservice.instance (). waituntillistening (); /* Update the local heartbeat counter.
                */Endpointstatemap.get (fbutilities.getbroadcastaddress ()). Getheartbeatstate (). Updateheartbeat (); if (logger.istraceenabled ()) Logger.trace ("My heartbeat are Now" + endpointstatemap.get (Fbuti
                Lities.getbroadcastaddress ()). Getheartbeatstate (). Getheartbeatversion ());
                Final list<gossipdigest> gdigests = new arraylist<gossipdigest> ();

                Gossiper.instance.makeRandomGossipDigest (gdigests); if (gdigests.size () > 0) {Gossipdigestsyn digestsynmessage = new Gossipdigestsyn (D Atabasedescriptor.getclustername (), Databasedes Criptor.geTpartitionername (), gdigests); Messageout<gossipdigestsyn> message = new Messageout<gossipdigestsyn> (MessagingService.Verb.GOSSIP_
                                                                                          Digest_syn, Digestsynmessage,
                    Gossipdigestsyn.serializer); /* Gossip to some random Live member/Boolean gossipedtoseed = Dogossiptolivemember (

                    message); /* Gossip to some unreachable/some probability to check if it's back/*/dogossiptounre

                    Achablemember (message);  /* Gossip to a seed if we did does so above, or we have seen less nodes than there are. This prevents partitions where each group of the nodes is onlY gossiping to a subset of the seeds. The most straightforward check would is to check this all seeds have been verified either as Li  ve or unreachable. To avoid so computation each round, we reason That:either all the live n Odes are seeds, in which case non-seeds that come online would introduce themselves to a He ring by definition, or there are at least one Non-seed node in the list, in which case eventuall
                       Y someone would gossip to it, and then does a gossip to a random seed from the

                       Gossipedtoseed check. CASSANDRA-150 for more exposition. */if (!gossipedtoseed | | liveendpoints.size () < Seeds.size ()) Dogossiptose

                    Ed (message);
                Dostatuscheck (); Catch(Exception e)
            {Logger.error ("Gossip error", e); }
        }
    }

From the above code can be seen gossiper. instance. Makerandomgossipdigest (gdigests); used to generate gossipdigest, and then invoke Dogossiptolivemember (message) Sends message messages to the online node, and then invokes the Dogossiptounreachablemember (message) method to send messages to the offline node to satisfy the if (!gossipedtoseed | | Liveendpoints.size () < Seeds.size ()) invokes the Dogossiptoseed () method to send a message to the seed node.

The Dogossiptolivemember method calls the Sendgossip () method,

  Private Boolean Dogossiptolivemember (messageout<gossipdigestsyn> message)
    {
        int size = Liveendpoints.size ();
        if (size = = 0) return
            false;
        return SENDGOSSIP (message, liveendpoints);
    

Private Boolean Sendgossip (messageout<gossipdigestsyn> message, set<inetaddress> epset)
    {
        List <InetAddress> liveendpoints = immutablelist.copyof (epset);
        
        int size = Liveendpoints.size ();
        if (Size < 1) return
            false;
        /* Generate a random number from 0-> size *
        /INT index = (size = = 1)? 0:random.nextint (size);
        InetAddress to = liveendpoints.get (index);
        if (logger.istraceenabled ())
            Logger.trace ("Sending a Gossipdigestsyn to {}", to);
        Messagingservice.instance (). Sendoneway (message, to);
        Return Seeds.contains (to);
    }

The Sendgossip () method is to randomly select a node from the liveendpoints and then send the message, while the processing of the SYN message takes place in the Gossipdigestsynverbhandler class, and the processing logic looks like this:

Dosort (gdigestlist);
        list<gossipdigest> deltagossipdigestlist = new arraylist<gossipdigest> ();
        Map<inetaddress, endpointstate> deltaepstatemap = new hashmap<inetaddress, endpointstate> ();
        Gossiper.instance.examineGossiper (Gdigestlist, Deltagossipdigestlist, Deltaepstatemap);
        Logger.trace ("Sending {} digests and {} deltas", Deltagossipdigestlist.size (), deltaepstatemap.size ()); messageout<gossipdigestack> gdigestackmessage = new Messageout<gossipdigestack> (
                                                                                        MessagingService.Verb.GOSSIP_DIGEST_ACK,
                                                                                        New Gossipdigestack (Deltagossipdigestlist, Deltaepstatemap),
        Gossipdigestack.serializer);
        if (logger.istraceenabled ()) Logger.trace ("Sending a gossipdigestackmessage to {}", from); Messagingservice.instance (). sendonewAy (gdigestackmessage, from); 

Call the Dosort () method to sort the gdigestlist, and then MessageingService.instance.sendOneWay ();

Send the message SYN message, the SYN message processing is in the Gossipdigestackverbhandler class, the main processing code is as follows:

  Gossipdigestack gdigestackmessage = message.payload;
        list<gossipdigest> gdigestlist = Gdigestackmessage.getgossipdigestlist ();
        Map<inetaddress, endpointstate> epstatemap = Gdigestackmessage.getendpointstatemap ();

        Logger.trace ("Received ack with {} digests and {} states", Gdigestlist.size (), epstatemap.size ()); if (epstatemap.size () > 0) {/* Notify the failure detector * * Gossiper.instance.notify
            Failuredetector (EPSTATEMAP);
        Gossiper.instance.applyStateLocally (EPSTATEMAP); } if (Gossiper.instance.isInShadowRound ()) {if (logger.isdebugenabled ()) log
            Ger.debug ("Finishing shadow round with {}", from);
            Gossiper.instance.finishShadowRound (); Return  Don ' t bother doing anything else, we have what we came for}/* Get the state required to send to this
      Gossipee-construct Gossipdigestack2message * *  Map<inetaddress, endpointstate> deltaepstatemap = new hashmap<inetaddress, endpointstate> ();
            for (Gossipdigest gdigest:gdigestlist) {inetaddress addr = Gdigest.getendpoint ();
            Endpointstate localepstateptr = Gossiper.instance.getStateForVersionBiggerThan (addr, gdigest.getmaxversion ());
        if (localepstateptr!= null) deltaepstatemap.put (addr, localepstateptr); } messageout<gossipdigestack2> gdigestack2message = new Messageout<gossipdigestack2> (MessagingService . Verb.gossip_digest_ack2, New Go
                                                                                           SsipDigestAck2 (Deltaepstatemap),
        Gossipdigestack2.serializer);
        if (logger.istraceenabled ()) Logger.trace ("Sending a gossipdigestack2message to {}", from); Messagingservice.instance (). sEndoneway (Gdigestack2message, from); 

This part of the code is mainly Deltaepstatemap if the node's new state information, and then encapsulated into GossipDigestAck2 message, and then sent out. The processing of the ACK message is then in the Gossipdigestack2verbhandler class,

The above code is for the ACK message processing,

Gossiper. instance. applystatelocally (Remoteepstatemap); his role is to update the state information of the node.

Org.apache.cassandra.locator

The Iendpointsnitch interface encapsulates methods such as acquiring racks, obtaining data centers, sorting by given node distances, and so on.

Inheritance structure for replicas:

Three copy policies are similar, for example: a copy of the key can be partitioner to find token value, and then according to token and copy placement policy to find all the nodes that put the replica, The call to the Getcachedendpoints () method is to first look up from the cache, verify the version in the cache, and obtain it from the cache if it is updated, otherwise call the Getnaturalendpoints () method to get all the nodes, and the sequence diagram of the method call is as follows:

As you can see from the code above, the way Simplestrategy handles nodes that are stored by token is to first tokenmetadata start with a given token and then find nodes in order from big to small, If it is found that the node is not added to the endpoints, the number of nodes is equal to the replica factor. This is exactly the same strategy as the Simplestrategy storage replica, where the first copy is placed on the first node that is larger than the key's token value, and then the others are placed on the node in the clockwise direction.

Networktopologystrategy deal with the method and simplestrategy consistent, not much talk about it.

4 Org.apache.cassandra.service

 

The Cassandradaemon class is responsible for Cassandra-initiated classes, and Cassandra Boot can do many things, such as initializing various configurations, loading keyspace, and compaction compression, and whether Bootstrap is needed. The following is the main sequence diagram when Cassandra starts.

The above boot process is focused on the first time the node to join the cluster needs bootstrap (non-seed node), the bootstrap process is as follows: first of all to determine whether the node needs bootstrap, the criteria are judged by three, one is the configuration item Auto_bootstrap is true, Second, judge whether the bootstrap is complete. The third is whether the node is a seed node, and if the configuration item is Auto_strap True and the bootstrap process does not end and the node is not a seed node, the token value of the node is obtained by invoking the method Bootstraptokens () To determine whether the value of the Initialtokens is greater than 0, and if the prisoner is greater than 0, convert the Initialtokens value to the token type to join tokens and return. Otherwise, the value of the numtokens is judged to be greater than 1 o'clock the Getrandomtokens method gets the partition, the partition randomly generates the token value, and if the token ' value does not exist in the Tokenmetadata, the tokens is added and returned. The main code looks like this:

Org.apache.casandra.thrift

 

Cassandraserver class, Cassandra the thrift server that responds to the client, Inherit the Cassandra.iface interface, consistent with the client, the client each send a request, the server side will be processed in the same way, because the method processing process is basically the same, so give an example: Get (Bytebufferkey, Columnpathcolumn _path, Consistencylevel Consistency_level) method gets the column, which is the following code,

The client queries the data to call the Get () method, the Get method calls the Send_get () method to send the request to the server, creates the Get_args object in the Send_get () method, sets the parameters, and then calls the Sendbase method to send to the server side through various protocols, The server-side calls the corresponding Get method () to process the request,

After the server end is processed, the Columnorsupercolumn object is returned to the client, and the client invokes the Receivebase method to receive the object.

Org.apache.cassandra.config

The Cfmetadata class is information about columnfamily, and the Config class is about Cassandra configuration information. This information can be obtained from the Cassandra configuration file Cassandra.yaml, Databasedescriptor can obtain information about the system, which is the information in the Config class that has been validated by some conditions.

Schema class Keyspace information about Cassandra and so on, modify Keyspace must modify schema information,

Yamlconfigurationloader class Loads configuration file Cassandra.yaml information,

The above code is to load Cassandra.yaml file information coexist to the corresponding attributes.

Org.apache.cassandra.cli

 

The Climain class is primarily a client that compiles and generates Cassandra Cassandra.

Clisessionstate stores information about the status of the Cassandra connection, including the name of the server, thrift client, user name, and password.

Org.apache.cassandra.auth

 

CAssandra's authorization mechanism, Cassandra's various permissions are set here. By assigning certain permissions to the user to restrict the user's behavior, the permission class also has the user's privileges, create,modify,drop,select,alter, and so on.

Org.apache.cassandra.cache

Cassandra Cache settings, this class does not look closely.