Build a storm development environment on a well-zookeeper cluster

Source: Internet
Author: User

Reference documents
Http://www.cnblogs.com/panfeng412/archive/2012/11/30/how-to-install-and-deploy-storm-cluster.html
Follow this document to install the appropriate support software

Reference documents
http://blog.csdn.net/u011362822/article/details/42463695
Remind me of a zeromq./configure error

Reference documents
http://uohzoaix.github.io/studies/2014/04/15/stormCluster/
Remind me of two error handling when installing JZMQ

Configuring in ~/.BASHRC
#storm
Export Storm_home= "/home/jason/stormprocessing/apache-storm-1.0.1"
Export path= $STORM _home/bin: $PATH

Configure this in the storm's configuration file

Storm.zookeeper.servers:     -"192.168.60.129"     -"192.168.60.132"     -"192.168.60.133" Storm.local.dir: "/ Home/jason/stormprocessing/apache-storm-1.0.1/workdir "Nimbus.host:" 192.168.60.129 "Supervisor.slots.ports:     -6700    -6701    -6702    -6703

The format of this configuration file is more stringent and should be carefully

In one of the nodes, run the following command to make this node Nimbus:
Nimbus: Run "bin/storm Nimbus >/dev/null 2>&1 &" On the Storm master node to launch the Nimbus daemon, and put it into the background execution;

Run the following command on the other nodes to become supervisor:
Supervisor: Run "bin/storm Supervisor >/dev/null 2>&1 &" On the Storm's various work nodes to start the Supervisor daemon, and put it into the background execution;

Run the following command in Nimbus to see the UI monitoring:
UI: Run "Bin/storm UI >/dev/null 2>&1 &" On the Storm master node to start the UI daemon and put it in the background, which can be done via Http://{nimbus host after startup. : 8080 Observe the usage of worker resources in the cluster, the running status of topologies, and other information.

[small summary]    bin/storm Nimbus >/dev/null2>&1 &    bin/ Storm UI >/dev/null2>&1 &    bin/storm Supervisor >/dev/ NULL 2>&1 &    storm/bin/storm logviewer >/dev/null2 >&1 &  

https://192.168.60.129:8080/
Found nothing.

In fact, my backstage program, did not run the error exits.
In order to see the problem of error
I'm not running in the background anymore.

[Email protected]:~/stormprocessing/apache-storm-1.0.1/bin$ Storm Nimbustraceback (most recent call Last): File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line766,inch<module>Main () File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line763,inchMain (Commands.get (COMMAND, Unknown_command)) (*ARGS) File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line526,inchNimbus jvmopts= Parse_args (Confvalue ("nimbus.childopts", cppaths)) +[File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line144,inchConfvalue P= Sub. Popen (Command, stdout=Sub. PIPE) File"/usr/lib/python2.7/subprocess.py", line710,inch__init__ Errread, Errwrite) File"/usr/lib/python2.7/subprocess.py", line1327,inch_execute_child raise Child_exceptionoserror: [Errno8] Exec format error File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line766,inch<module>Main () File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line763,inchMain (Commands.get (COMMAND, Unknown_command)) (*ARGS) File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line569,inchSupervisor Jvmopts= Parse_args (Confvalue ("supervisor.childopts", cppaths)) +[File"/home/jason/stormprocessing/apache-storm-1.0.1/bin/storm.py", line144,inchConfvalue P= Sub. Popen (Command, stdout=Sub. PIPE) File"/usr/lib/python2.7/subprocess.py", line710,inch__init__ Errread, Errwrite) File"/usr/lib/python2.7/subprocess.py", line1327,inch_execute_child Raise Child_exception

You can see a problem like permissions or Python.


Then close the Ubuntu firewall
UFW Disable
UFW enable//Turn the firewall back on

Because I'm not installing python myself with Python
So just configure it.
#python
Export Python_home= "/usr/lib/python2.7"
Export path= $PYTHON _home: $PATH

But there are a lot of mistakes that don't work.
and found that although these required scripts are under the $path path
But I still can't find the command when I do it with sudo.
Sometimes zookeeper doesn't start at all.

So every run, like,
sudo./zkserver.sh start
sudo./zkserver.sh status
sudo./storm Nimbus
sudo./storm Supervisor
You need to run to the appropriate bin path.

Can not run all restart, and then you will see the Dear UI interface

Start the topology
Storm jar Allmycode.jar org.me.MyTopology arg1 arg2 arg3
Allmycode.jar is a jar package that contains the topology implementation code
The main method of Org.me.MyTopology is the entrance to the topology.
Arg1, arg2, and ARG3 parameters that need to be passed in for org.me.MyTopology execution
Stop topology
Storm Kill {Toponame}
where {toponame} is the name of the topology task specified when topology commits to the storm cluster.

To actually see the effect
Reference documents
http://blog.csdn.net/loloxiaoz3/article/details/19294289
Follow the above method to download an example program
Using MAVEN to compile Storm_starter source code
Download: Git clone https://github.com/nathanmarz/storm-starter.git
Compilation: sudo mvn-f m2-pom.xml compile
Pack: sudo mvn-f m2-pom.xml package
Delete: sudo mvn-f m2-pom.xml clean
Mvn-f means forcing the m2-pom to be used to compile

But mvn downloaded the jar package is slow, not yet see the end of the sign, it is estimated that the UI is not visible today.


Submit example in Storm_starter to the Storm cluster
Submit topology:
sudo storm jar Storm-starter-0.0.1-snapshot-jar-with-dependencies.jar storm.starter.WordCountTopology WordCount
Terminate topology:
Storm Kill WordCount

Build a storm's development environment on a well-zookeeper cluster

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.