Storm/jstorm's Topologybuilder Source Reading

There is a class in strom/jstorm that is especially important for building topology, and this class is Topologybuilder.
Let's look at a simple example:

 public Static voidMain (string[] Args)throwsalreadyaliveexception, invalidtopologyexception {topologybuilder builder=NewTopologybuilder (); Builder.setspout ("input",NewRandomsentencespout (), 2); Builder.setbolt ("bolt_sentence",NewSplitsentencebolt (), 2). shufflegrouping ("input"); //local Mode: The main use for debuggingLocalcluster cluster =NewLocalcluster (); System.out.println ("start wordcount"); Cluster.submittopology ("word count", conf, builder.createtopology ()); }

In the Main method above, first create the Topologybuilder object, then set up the created spout node and bolt node, and use random grouping (shufflegrouping) to connect spout and bolt nodes together to form Topology.

How did the Topologybuilder do it? Please see below Topologybuilder source code:

/*** Topologybuilder is a tool class for building topology **/ public classTopologybuilder {/*** Defines the class member variable _bolts to hold all bolt objects of the Irichbolt type*/    Privatemap<string, irichbolt> _bolts =Newhashmap<string, irichbolt>(); /*** Defines the class member variable _spouts to hold all spout objects of the Irichspout type*/    Privatemap<string, irichspout> _spouts =Newhashmap<string, irichspout>(); /*** Defines the class member variable _commons, which holds all bolts and spout objects*/    Privatemap<string, componentcommon> _commons =Newhashmap<string, componentcommon>(); //private map<string, map<globalstreamid, grouping>> _inputs = new hashmap<string, map< globalstreamid, grouping>> ();    Privatemap<string, statespoutspec> _statespouts =Newhashmap<string, statespoutspec>(); /*** Build Stormtopology objects based on incoming bolts and spout objects *@return     */ publicstormtopology createtopology () {Map<string, bolt> boltspecs =Newhashmap<string, bolt>(); Map<string, spoutspec> spoutspecs =Newhashmap<string, spoutspec>();  for(String boltid: _bolts.keyset ()) {//gets the corresponding bolt object from the _bolts according to BoltidIrichbolt Bolt =_bolts.get (boltid); //sets the streams of the corresponding Componentcommon object (the field List of the output and whether it is a direct Stream) property valueComponentcommon common =Getcomponentcommon (boltid, bolt); /*** The Bolts object is serialized to get an array, and then the Bolt object is created, so all bolts in Stormtopology is the byte array that is obtained after the object is Serialized. */boltspecs.put (boltid,NewBolt (componentobject.serialized_java (utils.javaserialize (Bolt)), common)); }         for(String spoutid: _spouts.keyset ()) {//gets the corresponding spout object from the _spouts according to Spoutidirichspout spout =_spouts.get (spoutid); //sets the streams of the corresponding Componentcommon object (the field List of the output and whether it is a direct Stream)Componentcommon common =Getcomponentcommon (spoutid, spout); /*** The Spout object is serialized to get the array, and then the Spoutspec object is created, so all spouts in Stormtopology is the byte array that is obtained after the object is Serialized. */spoutspecs.put (spoutid,Newspoutspec (componentobject.serialized_java (utils.javaserialize (spout)), common)); }        //encapsulates all of the components set above into the Stormtopology object, and finally commits to the cluster to run        return NewStormtopology (spoutspecs, boltspecs,Newhashmap<string, statespoutspec>()); }    /*** The following methods define the Setbolt method and its overloaded methods*/    /*** Define a new Bolt in this topology with only single thread parallelism * Other components that want to consume the output of this bolt will reference this ID*/    publicboltdeclarer setbolt (String id, irichbolt Bolt) {returnSetbolt (id, bolt,NULL); }    /*** A bolt that defines a specified number of degrees of parallelism for this topology*/     publicboltdeclarer setbolt (String id, irichbolt bolt, number Parallelism_hint) {//detects if the incoming component ID is uniqueValidateunusedid (id); //Generating Common ObjectsInitcommon (id, bolt, parallelism_hint);        _bolts.put (id, bolt); return NewBoltgetter (id); }     publicboltdeclarer setbolt (String id, ibasicbolt Bolt) {returnSetbolt (id, bolt,NULL); }     publicboltdeclarer setbolt (String id, ibasicbolt bolt, number Parallelism_hint) {/*** This method uses the Basicboltexecutor wrapper (encapsulation) incoming Ibasicbolt object * To achieve the tracking of messages in the Basicboltexecutor*/        returnSetbolt (id,NewBasicboltexecutor (bolt), parallelism_hint); }  /*** The following methods define the Setspout method and its overloaded methods*/     publicspoutdeclarer setspout (String id, irichspout spout) {returnSetspout (id, spout,NULL); }     publicspoutdeclarer setspout (String id, irichspout spout, number Parallelism_hint) {//detects if the ID of the input is unique and throws an exception if it already existsValidateunusedid (id); /*** Build Componentcommon objects and initialize them, and put them into _commons (defined above)*/Initcommon (id, spout, parallelism_hint);        _spouts.put (id, spout); return NewSpoutgetter (id); }     publicspoutdeclarer setspout (String id, icontrolspout spout) {returnSetspout (id, spout,NULL); }     publicspoutdeclarer setspout (String id, icontrolspout spout, number Parallelism_hint) {returnSetspout (id,NewControlspoutexecutor (spout), parallelism_hint); }     publicboltdeclarer setbolt (String id, icontrolbolt bolt, number Parallelism_hint) {returnSetbolt (id,NewControlboltexecutor (bolt), parallelism_hint); }     publicboltdeclarer setbolt (String id, icontrolbolt Bolt) {returnSetbolt (id, bolt,NULL); }     public voidsetstatespout (String id, irichstatespout statespout) {setstatespout (id, statespout,NULL); }     public voidsetstatespout (String id, irichstatespout statespout, number Parallelism_hint) {validateunusedid (id); //Todo:finish    }    /*** Detects if the input ID is unique *@paramID*/    Private voidValidateunusedid (String Id) {if(_bolts.containskey (ID)) {Throw NewIllegalArgumentException ("Bolt had already been declared for id" +id); }        if(_spouts.containskey (ID)) {Throw NewIllegalArgumentException ("Spout have already been declared for id" +id); }        if(_statespouts.containskey (ID)) {Throw NewIllegalArgumentException ("state spout have already been declared for id" +id); }    }    PrivateComponentcommon getcomponentcommon (String id, IComponent component) {componentcommon ret=NewComponentcommon (_commons.get (id)); Outputfieldsgetter Getter=NewOutputfieldsgetter ();        Component.declareoutputfields (getter);        Ret.set_streams (getter.getfieldsdeclaration ()); returnret; }    /*** Defines the Initcommon method, used to initialize the variable Commonentcommon object, and assigns the class member variable _commons to initialize the work Done: set the degree of parallelism and some other configuration *@paramID *@paramComponent *@paramparallelism*/Private voidInitcommon (String id, IComponent component, number Parallelism) {componentcommon Common=NewComponentcommon (); //set the source and grouping of message flowsCommon.set_inputs (Newhashmap<globalstreamid, grouping>()); if(parallelism! =NULL) {            //setting the degree of parallelismCommon.set_parallelism_hint (parallelism.intvalue ()); } Else {            //The default is 1 if the degree of parallelism is not set manuallyCommon.set_parallelism_hint (1); } MAP conf=component.getcomponentconfiguration (); if(conf! =NULL)            //Set configuration parameters for a componentcommon.set_json_conf (jsonvalue.tojsonstring (conf));    _commons.put (id, common); }}

As you can see from the Topologybuilder class above, this class provides methods for creating stormtopology and some data source nodes and related settings for processing nodes.

There are methods for storing bolt objects and spout objects, of course, where the code for grouping is not Written. In fact, This class is used to set up spout nodes and Bolt nodes,

The spout and bolt nodes are connected together to form a topological structure by means of grouping.

Storm/jstorm's Topologybuilder Source Reading