Storm installs debugging under Windows system

Basic knowledge

Storm is a distributed, reliable, fault-tolerant data flow processing system. It delegates work tasks to different types of components, each of which handles a simple, specific task. The input stream of the storm cluster is managed by a component called spout, which spout the data to the bolt, which either saves the data to a certain memory or passes the data to other bolts. As you can imagine, a storm cluster is the conversion of spout data from one series of bolts to another.

Concept Explanation:

• Spout: Reads the raw data to provide data for the bolt.
• Bolt: Receives data from spout or other bolts and processes the data, and the result can be used as the data source or the final result of other bolts.
• Nimbus: The daemon of the master node that is responsible for distributing tasks for the work node.
• Topology: Topological structure, a task unit of storm.
• Define field (s): Defines the domain, provided by the spout or bolt, that is received by the bolt.

Installation:

Install Zookeeper:

• Download Zookeeper:http://zookeeper.apache.org/releases.html#download.
• Extract the zookeeper-3.4.7.tar.gz to the directory, D:\bin\zookeeper-3.4.7.
• Copy the file D:\bin\zookeeper-3.4.7\conf\zoo_sample.cfg to zoo.cfg in the same directory.
• Modify the contents of the zoo.cfg as needed.
• Run D:\bin\zookeeper-3.4.7\bin\zkServer.cmd.

Install storm:

• Download storm:http://storm.apache.org/downloads.html.
• Extract the apache-storm-0.10.0.tar.gz to the directory, D:\bin\apache-storm-0.10.0.
• Run D:\bin\storm.cmd Nimbus
• Run D:\bin\storm.cmd Supervisor
• Running the D:\bin\storm.cmd UI
• With http://localhost:8080 you can open the Storm Web UI and see the state of the storm cluster.

Run error Handling:

• If you report a JAVA version issue, install jdk1.7 and set the java_home correctly.
• Set the environment variable storm_home=d:\bin\apache-storm-0.10.0.
• Set the environment variable storm_conf_dir=%storm_home%\conf.
• "Java.net.malformedurlexception Unknown protocol D" error occurs, set the environment variable storm_logback_configuration_file=file:///d:/bin/ Apache-storm-0.10.0/log4j2/cluster.xml
• If the "Unable to delete file" error is reported, it is a bug,0.8.2 version of storm that has not been resolved, this error is thrown in Cluster.shutdown (); This issue does not occur with version 0.10.0.

The Storm Web UI interface is as follows:

Start with a simple example

• Class Wordreader: Reads the text content from the file, emit to bolts for each row read.
• Class Wordnormalizer: Splits a line of text content by word.
• Class WordCounter: Counts the number of occurrences of a word.
• Class Topologymain: Create a topology.

Source code See File attachment: Examples-ch02-getting_started.zip.

Native debugging

In the development tool, run Topologymain directly to begin the experience Strom

Publish the topology to the cluster

You need to change the localcluster to Stormsubmitter and call the Submittopology method

    Localcluster cluster = new Localcluster ();    Cluster.submittopology ("Count-word-topology-with-refresh-cache", conf,     //builder.createtopology ());    Stormsubmitter.submittopology ("Count-word-topology-with_refresh-cache", conf,            builder.createtopology ());    Thread.Sleep (+);    Cluster.shutdown ();

Next, compress the source code into a jar package, run the Storm Client command, and submit the topology to the cluster. If you have already used Maven, you only need to go to the source directory at the command line: MVN package.

Now that you have generated a jar package, use the Storm Jar command to submit the topology. Command format: Storm jar Allmycode.jar org.me.MyTopology arg1 arg2 arg3.

For this example, run under the Topology Engineering directory:

Storm Jar Topologies-0.0.1-snapshot.jar Countword. Topologymain Src/main/resources/words.txt

With these commands, you have the topology published on the cluster.

If you want to stop or kill it, run:

Storm Kill Count-word-topology-with-refresh-cache

Resources

Getting Started with Storm: Http://ifeve.com/wp-content/uploads/2014/03/Getting-Started-With-Storm-Jonathan-Leibiusky-Gabriel-E_1276.pdf
Official Document: Http://storm.apache.org/documentation.html

Description

1. The folder location in this document should be modified as needed.
2. The software version used in this document is not necessarily up-to-date.

Storm installs debugging under Windows system