Installation of Apache Storm

Original: http://storm.praveendeshmane.co.in/storm/storm-0-10-0-installation-on-ubuntu-14-04.jsp

------------------------------------------

Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure Progra Mming language. Originally created by Nathan Marz and team at Backtype, the project is open sourced after being acquired by Twitter. It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed PR Ocessing of streaming data. The initial release was on September 2011.

A Storm application is designed as a "topology" in the shape of a directed acyclic graph (DAG) with spouts and bolts actin G as the graph vertices. Edges on the graph is named streams and direct data from one node to another. Together, the topology acts as a Data transformation pipeline. At a superficial level the general topology structure was similar to a MapReduce job with the main difference being that D ATA is processed in real time as opposed to in individual batches.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system.

2) Apcahe ZooKeeper Pre installed (how to install ZooKeeper on Ubuntu 14.04)

3) Apache Storm 0.10.0 Software (Download here)

Storm installation Steps

Step 1- installing Java 7. Open a terminal (CTRL + ALT + T) and type the following sudo command.

$ sudo apt-get install OPENJDK-7-JDK

Check it installed properly or not.

$ java-version

Step 2- Edit $HOME/.BASHRC file by adding the Java path.

$ sudo gedit $HOME/.BASHRC

$HOME/.BASHRC File

Export JAVA_HOME=/USR/LIB/JVM/JAVA-7-OPENJDK-AMD64

Step 3- Reload your changed $HOME/.BASHRC settings

$ source $HOME/.BASHRC

Step 4- Creating Storm directory. Open a terminal (CTRL + ALT + T) and type the following $ sudo command.

$ sudo mkdir/usr/local/storm

Step 5- Change the ownership and permissionsof the Directory/usr/local/storm. Here ' HDUser ' are an Ubuntu username.

$ sudo chown-r hduser/usr/local/storm

$ sudo chmod-r 755/usr/local/storm

Step 6- Change the directory To/home/hduser/desktop , in my case the downloaded apache-storm-0.10.0.tar.gz file is In/home/hduser/desktop folder. For you it might is in /downloads folder check it.

$ cd/home/hduser/desktop/

Step 7- Untar the apache-storm-0.10.0.tar.gz file.

$ tar xzf apache-storm-0.10.0.tar.gz

Step 8- Move the contents of apache-storm-0.10.0 folder To/usr/local/storm

$ mv apache-storm-0.10.0/*/usr/local/storm

Step 9- Change the directory To/usr/local/storm

$ cd/usr/local/storm

Step Ten- Create/data,/data/nimbus and/data/supervisor directories.

$ sudo mkdir/usr/local/storm/data$ sudo mkdir/usr/local/storm/data/nimbus$ sudo mkdir/usr/local/storm/data/ Supervisor

Step One- Change the ownership and permissions of the Directory/data,/data/nimbus and/data/supervisor directories. Here ' HDUser ' are an Ubuntu username.

$ sudo chown-r hduser/usr/local/storm/data$ sudo chown-r hduser/usr/local/storm/data/nimbus$ sudo chown-r hduser/usr /local/storm/data/supervisor

$ sudo chmod-r 755/usr/local/storm/data$ sudo chmod-r 755/usr/local/storm/data/nimbus$ sudo chmod-r 755/usr/local/st Orm/data/supervisor

Step- Change the directory to/usr/local/storm/conf

$ cd/usr/local/storm/conf

Step- Edit storm.yaml file.

$ gedit Storm.yaml

Step- ADD These below lines to Storm.yaml file. Save and close.

Storm.zookeeper.servers:-"localhost" storm.local.dir: "/usr/local/storm/data" nimbus.host: "localhost" nimbus.thrift.port:49627storm.zookeeper.port:2181supervisor.slots.ports:-6700-6701-6702-6703

Step- Start ZooKeeper. Open a new terminal (CTRL + ALT + T) and start zookeeper.

$/usr/local/zookeeper/bin/zkserver.sh Start

Step- Open a new terminal (CTRL + ALT + T). Change the directory To/usr/local/storm

$ cd/usr/local/storm

Step- Start Nimbus

$./bin/storm Nimbus

Step- Open a new terminal (CTRL + ALT + T). Change the directory To/usr/local/storm

$ cd/usr/local/storm

Step- Start Supervisor

$./bin/storm Supervisor

Step- Open a new terminal (CTRL + ALT + T). Change the directory To/usr/local/storm

$ cd/usr/local/storm

Step- Start Web UI

$./bin/storm UI

Step- Check the Web UI here. Open a browser and type the following URL.

http://localhost:8080

Share this blog post and follow me for latest updates on

Installation of Apache Storm