Storm's 6:api

(i) An example
This example uses storm to run the classic WordCount program with the following topology:
Sentence-spout->split-bolt->count-bolt->report-bolt
Complete the production of sentences, split the words, the number of words, the statistical results of the output
Complete code See Https://github.com/jinhong-lu/stormdemo
The following is an analysis of the critical code.

1. Create spout

public class Sentencespout extends Baserichspout {private Spoutoutputcollector collector;    private int index = 0; Private string[] sentences = {"When I am young I ' d listen to the Radio", "Waiting for my favorite songs", "Wh En they played i ' D Sing along "," it Make me Smile "," those were such happy times and isn't so long ago            "How I wondered where they ' d gone", "but they ' re-back again just like a long lost friend",             "All the songs I love So Well", "every Shalala every wo ' wo", "Still shines.", "Every Shing-a-ling-a-ling",    "That's they ' re starting", "to Sing So Fine"}; public void Open (Map conf, topologycontext context, Spoutoutputcollector collector) {this.collector = c    Ollector;    } public void Declareoutputfields (Outputfieldsdeclarer declarer) {Declarer.declare (new fields ("sentence")); } public void Nexttuple () {This.collector.emit (new Values (senTences[index]));        index++;        if (index >= sentences.length) {index = 0;        } try {Thread.Sleep (1);        } catch (Interruptedexception e) {//e.printstacktrace (); }    }}

In the above class, the contents of a string array are sent out line by row, the main methods are:
(1) The Open () method completes the initialization of the spout, similar to the prepare () method of the Bolt
(2) Declareoutputfileds () defines the field name of the sent content as the number of fields, and the method name in Bolt.
(3) The Nexttuple () method is the same for every data that needs to be processed, and the executor () method of the bolt is similar. It is the core of the entire logic process, sending data to the next node in the topology through the emit () method.

2. Create Split-bolt

public class Splitsentencebolt extends baserichbolt{    private outputcollector collector;    public void Prepare (Map stormconf, Topologycontext context,            Outputcollector collector) {        This.collector = Collector;    }        public void Declareoutputfields (Outputfieldsdeclarer declarer) {        Declarer.declare (new fields ("word"));    }    public void execute (Tuple input) {        String sentence = Input.getstringbyfield ("sentence");        string[] Words = Sentence.split ("");        for (String word:words) {            this.collector.emit (new Values (word));            SYSTEM.OUT.PRINTLN (Word);}}    }

The meaning of the three methods is similar to spout, which splits the received sentences according to a space, splits the words into one, and then sends the words out individually.
Input.getstringbyfield ("sentence") can obtain the corresponding content based on the keywords sent by the previous node.

3. Create Wordcount-bolt

public class Wordcountbolt extends baserichbolt{    private outputcollector collector;    Private map<string,long> counts = null;    public void Prepare (Map stormconf, Topologycontext context,            Outputcollector collector) {        This.collector = Collector;        this.counts = new hashmap<string, long> ();    }    public void Declareoutputfields (Outputfieldsdeclarer declarer) {        declarer.declare (the new fields ("word", "count"));    Public    void Execute (Tuple input) {        String word = Input.getstringbyfield ("word");        Long count = This.counts.get (word);        if (count = = null) {            count = 0L;        }        count++;        This.counts.put (Word, count);        This.collector.emit (New Values (Word,count));        System.out.println (count);}    }

This class will receive the number of word to be counted, and send out the results.
This bolt sends 2 filed:
Declarer.declare (New fields ("word", "count"));
This.collector.emit (New Values (Word,count));

4. Create Report-bolt

public class Reportbolt extends baserichbolt{private map<string, long> counts; public void Prepare (Map stormconf, Topologycontext context, Outputcollector collector) {this.counts =    New Hashmap<string,long> ();        } public void Declareoutputfields (Outputfieldsdeclarer declarer) {} public void execute (Tuple input) {        String Word = Input.getstringbyfield ("word");        Long count = Input.getlongbyfield ("Count");    Counts.put (Word, count);        } public void Cleanup () {System.out.println ("Final output");        Iterator<entry<string, long>> iter = Counts.entryset (). Iterator ();            while (Iter.hasnext ()) {entry<string, long> Entry = Iter.next ();            String word = (string) entry.getkey ();            Long Count = (long) entry.getvalue ();        SYSTEM.OUT.PRINTLN (Word + ":" + count);    } super.cleanup (); }        }
 

This class will output the data received from the Wordcount-bolt.
The result is placed in a map, and when Topo is closed, the cleanup () method is called, and the contents of the map are output.

5. Create topo

public class Wordcounttopology {private static final String sentence_spout_id = "Sentence-spout";    private static final String split_bolt_id = "Split-bolt";    private static final String count_bolt_id = "Count-bolt";    private static final String report_bolt_id = "Report-bolt";    private static final String Topology_name = "Word-count-topology";        public static void Main (string[] args) {sentencespout spout = new Sentencespout ();        Splitsentencebolt Splitbolt = new Splitsentencebolt ();        Wordcountbolt Countbolt = new Wordcountbolt ();        Reportbolt Reportbolt = new Reportbolt ();        Topologybuilder builder = new Topologybuilder ();        Builder.setspout (sentence_spout_id, SPOUT);        Builder.setbolt (split_bolt_id, Splitbolt). shufflegrouping (sentence_spout_id);        Builder.setbolt (count_bolt_id, Countbolt). fieldsgrouping (split_bolt_id, new fields ("word")); Builder.setbolt (report_bolt_id, Reportbolt). GlobalgrouPing (count_bolt_id);        Config conf = new config ();            if (Args.length = = 0) {localcluster cluster = new Localcluster ();            Cluster.submittopology (topology_name, Conf, builder.createtopology ());            try {thread.sleep (10000);            } catch (Interruptedexception e) {} cluster.killtopology (topology_name);        Cluster.shutdown ();            } else {try {stormsubmitter.submittopology (args[0], conf,builder.createtopology ());            } catch (Alreadyaliveexception e) {e.printstacktrace ();            } catch (Invalidtopologyexception e) {e.printstacktrace (); }        }    }}

The key steps are:
(1) Create Topologybuilder and specify spout and bolts for this builder
Builder.setspout (sentence_spout_id, SPOUT);
Builder.setbolt (split_bolt_id, Splitbolt). Shufflegrouping (
SENTENCE_SPOUT_ID);
Builder.setbolt (count_bolt_id, Countbolt). fieldsgrouping (split_bolt_id,
New Fields ("word"));
Builder.setbolt (report_bolt_id, Reportbolt). Globalgrouping (
COUNT_BOLT_ID);
(2) Create a Conf object
Config conf = new config ();
This object is used to specify some topology-related properties, such as parallelism, Nimbus address, and so on.
(3) Create and run the topology, which uses 2 different ways
One is when there is no parameter, establish a localcluster, run directly on the local, after 10 seconds of running, shut down the cluster:
Localcluster cluster = new Localcluster ();
Cluster.submittopology (Topology_name, Conf,builder.createtopology ());
Thread.Sleep (10000);
Cluster.killtopology (Topology_name);
Cluster.shutdown ();
The second argument is that the topology is submitted to the cluster:
Stormsubmitter.submittopology (Args[0], conf,builder.createtopology ());
The first parameter is the name of the topology.

6. Local operation
Run directly in Eclipse and the output is visible in the console

7. Cluster operation
(1) Compiling and packaging
MVN Clean Compile
(2) Upload the compiled jar package to the Nimbus machine and
Storm Jar Com.ljh.storm.5_stormdemo Com.ljh.storm.wordcount.WordCountTopology Topology_name
Submit the topology to the cluster.

Storm's 6:api