Storm Starter Example rollingtopwords code learning

The author's description of this example: here.

Rollingtopwords implements the hottest topics in M-minute updates per N-minute update, such as refreshing a hot topic for the past 5 minutes every 1 minutes.

• Implementation ideas

• Spouts is responsible for pushing all topics (words), the same topic pushed to the same Rollingcountbolt task (using Storm's fieldsgrouping implementation)
• Rollingcountbolt receives the push topic and saves the number of occurrences of the topic. Every 1 minutes, Rollingcountbolt pushes the number of occurrences of each topic to the Intermediate processing node Intermediaterankingsbolt (also using Storm's fieldsgrouping implementation)
• Intermediaterankingsbolt holds a list of top N, refreshes the top N after receiving a push message (sort word based on the number of topics posted). The results are pushed to the final node Totalrankingsbolt every two seconds.
• Totalrankingsbolt selects the final top n after summarizing each top N received within 2 seconds.

• Key implementation Notes

• The Strom provides a timer-like mechanism that automatically sends a tuple to the bolt every time a specified period can be set in the bolt by the following code

@Overridepublic map<string, object> getcomponentconfiguration () {    map<string, object> conf = new Hashmap<string, object> ();    Conf.put (Config.topology_tick_tuple_freq_secs, emitfrequencyinseconds);    return conf;  }

Determines whether the system automatically sends Tuplepublic static Boolean istick (tuple tuple) {    return tuple! = null           && constants.system_ component_id  . Equals (Tuple.getsourcecomponent ())           && Constants.SYSTEM_TICK_STREAM_ID.equals ( Tuple.getsourcestreamid ());}

• To achieve the "refresh every 1 minutes in 5 minutes hot topic" requirements, 5 minutes will be divided into 5 segments, each minute for a paragraph. In this implementation the clock will be called a window for 5 minutes, and each paragraph is called a slot. You need to keep the number of topics in each paragraph in the system. Sum the preceding 5 minutes every 1 minutes, then discard the first paragraph of data. The above procedure is implemented by the Slidingwindowcounter class and the Slotbasedcounter class and is called in Rollingcountbolt.

• Doubts
  • This example does thread synchronization when Bolt calls a method in the rankings class, does it mean that storm's call to each task is not thread-safe?

Storm Starter Example rollingtopwords code learning