storm1.0.2 version Cluster build complete steps

storm1.0.2 version Cluster build complete steps

Storm Cluster Setup
Storm clusters are also made up of master and slave nodes.
Changes to the Storm version:
storm0.9.x
storm0.10.x
storm1.x
In the previous versions, Storm's core source code was made up of java+clojule.
storm2.x
Later this version is all in Java rewrite.
(Ali has rewritten the storm process very early, providing Jstorm, and later Jstorm also added to Apachestorm
Responsible for using Java to rewrite storm, which is the origin of the storm2.x version. ）
Attention:
In the storm0.9.x version, the storm cluster supports only one Nimbus node, the primary node being problematic.
After storm0.10.x, the storm cluster can support multiple Nimbus nodes, one of which is leader, which is responsible for really running and the rest for offline.
Master node (Control node Master) "Master node can have one or more"
Responsibilities: Responsible for distributing code, monitoring the execution of code.
Nimbus
UI: Can view the information of the cluster and the operation of topology
Logviewer: Because there will be multiple primary nodes, sometimes it is necessary to view the log information of the master node.
From node (worker node) "From node can have one or more"
Responsibilities: Responsible for generating worker processes and performing tasks.
Supervisor
Logviewer: You can view the run log of topology through the WebUI interface
Plan the deployment of your cluster
Master node:
192.168.1.100
Nimbus
Ui
Logviewer
192.168.1.101
Nimbus
Logviewer
From the node
192.168.1.101
Supervisor
Logviewer
192.168.1.102
Supervisor
Logviewer
Start deploying Storm Cluster (centos6.8):
The basic environment of the server needs to be confirmed before deploymentstorm cluster deployment relies on the following environments:

python2.6.6 and above "centos6.8 self-bringing"
jdk1.7 "Need to install"
SSH password-free login between cluster nodes "need to configure" Zookeeper "Storm cluster is dependent on zookeeper, need to install"

JDK installation "All nodes of the cluster need to be installed"
1: Download the jdk1.7 compression pack. Extract.
2: Configure the JAVA_HOME environment variable in the/etc/profile file
Export JAVA_HOME=/USR/LOCAL/JDK
Export path=.: Javahome/bin:j a V a H O M e/b i N:java_home/bin:path
SSH password-free login between cluster nodes
Suggest Baidu ssh password-free login
Zookeeper Installation "recommended to install a single-machine service first"
1: First download the Zookeeper installation package zookeeper-3.4.6.tar.gz
2: Upload to 192.168.1.100 this server's/usr/local directory
3: Unzip, rename
Cd/usr/local
TAR-ZXVF zookeeper-3.4.6.tar.gz
MV zookeeper-3.4.6 Zookeeper
4: Modify the configuration file
CD zookeeper/conf
MV Zoo_sample.cfg Zoo.cfg
VI zoo.cfg
Datadir=/usr/local/zookeeper/data
5: Start ZK standalone service
CD Zookeeper
bin/zkserver.sh start
Start installing Storm cluster
1: Download Storm's installation package apache-storm-1.0.2.tar.gz
2: Upload to 192.168.1.100 this server's/usr/local directory
3: Unzip, rename
TAR-ZXVF apache-storm-1.0.2.tar.gz
MV apache-storm-1.0.2 Storm
4: Modify the configuration file
CD storm/conf
VI Storm.yaml
Storm.zookeeper.servers:
-"192.168.1.100"
Nimbus.seeds: ["hadoop100", "hadoop101"]Note: It is best to specify the hostname when nimbus.seeds, otherwise the node information will be displayed repeatedly when querying the WebUI interface.

Storm.local.dir: "/usr/local/storm/tmp"
Supervisor.slots.ports:
-6700
-6701
-6702
-6703
Note: The format of the Yaml file, the attributes must be shelf, the indentation can use only spaces, and a tab cannot be used. 】
5: Copy the modified storm directory to all other master and slave nodes "master-slave node configuration exactly"
Scp-rq/usr/local/storm 192.168.1.101:/usr/local
Scp-rq/usr/local/storm 192.168.1.102:/usr/local
6: The process of starting the master-slave node separately
192.168.1.100
Nimbus:nohup bin/storm Nimbus >/dev/null 2>&1 &
Ui:nohup Bin/storm UI >/dev/null 2>&1 &
Logviewer:nohup bin/storm logviewer >/dev/null 2>&1 &
192.168.1.101
Nimbus:nohup bin/storm Nimbus >/dev/null 2>&1 &
Logviewer:nohup bin/storm logviewer >/dev/null 2>&1 &
Supervisor:nohup bin/storm Supervisor >/dev/null 2>&1 &
192.168.1.102
Supervisor:nohup bin/storm Supervisor >/dev/null 2>&1 &
Logviewer:nohup bin/storm logviewer >/dev/null 2>&1 &
7: Access the Storm's UI to view cluster information
http://192.168.1.100:8080/
Troubleshoot the problem:
1: If found unable to view log information in the WebUI interface
It could only be two problems.
One: Maybe the logviewer process didn't start.
Two: The host name and IP mapping relationship is not configured
2: The server's 8080 port is occupied causing the UI process to start failing
Modify the Storm.yaml file
ui.port:8081
The most complete property is configured in the Defaults.yaml file in Storm-core.jar

For more information on big data, videos and technical exchanges, please Dabigatran:

QQ Group No. 1:295,505,811 (full)

QQ Group No. 2:54,902,210

QQ Group No. 3:555,684,318