Twitter Storm installation configuration (cluster) Notes

The installation and configuration of the Twitter Storm standalone version have been clearly stated in the previous note:
 
This article mainly describes how to expand to a cluster based on the standalone version.
 
All the machines in the cluster need to install the tool software as needed in the standalone version:
 
Python, zookeeper, zeromq, jzmq, and storm
 
Install the tool one by one described in the installation of the standalone edition tutorial.
 
The difference is that it is configured. Here we configure three servers, hostA hostB hostC.
 
First, modify the configuration file of zookeeper:
 
Vim/usr/local/zookeeper/conf/zoo. cfg:
 
 
 
# The number of milliseconds of each tick
TickTime = 2000
# The number of ticks that the initial
# Synchronization phase can take
InitLimit = 10
# The number of ticks that can pass
# Sending a request and getting an acknowledgement
SyncLimit = 5
# The directory where the snapshot is stored.
DataDir =/tmp/zookeeper
# The port at which the clients will connect
ClientPort = 2181
 
Server.1 = hostA: 2888: 3888
Server.2 = hostB: 2888: 3888
Server.3 = hostC: 2888: 3888
 
The above configuration file can be the same on several machines.
 
Echo 1>/tmp/zookeeper/myid
 
This step is different on every computer.
 
For more information, see server. x = hostname: portNumber in zoo. cfg.
 
In my configuration file, hostA's echo is 1, hostB's echo is 2, and hostC's echo is 3.
 
 
 
Start the three machines respectively:
 
Bin/zkSever. sh start
 
This step is to start zookeeper on three machines, each machine has to do.
 
The following commands check the zookeeper status in the Cluster:
• # Echo stat | nc HOSTMASTER 2181
• # Echo con "| nc HOSTMASTER 2181
• # Echo dump | nc HOSTMASTER 2181
• # Echo wchs | nc HOSTMASTER 2181
• # Echo ruok | nc HOSTMASTER 2181
• # Bin/zkCli. sh. stat/
• # Bin/zkCli. sh ls/
 
Okay. The zookeeper cluster is configured. Next, configure storm:
 
Vim/usr/local/storm/conf/storm. yaml:
 

Storm. zookeeper. servers:
-"HostA"
-"HostB"
-"HostC"
 
Storm. zookeeper. port: 2181
 
Nimbus. host: "hostA"
 
Storm. local. dir: "/tmp/storm"
Storm. local. mode. zmq: true
 
Supervisor. slots. ports:
-6700
-6701
-6702
-6703
 
 
 
Note:
 
Storm. local. dir indicates the local directory used by storm.
 
Nimbus. host indicates that the machine is a master machine, that is, nimbus.
 
Storm. zookeeper. servers indicates which machines are zookeeper servers.
 
Storm. zookeeper. port indicates the zookeeper port number, which must be consistent with the port number configured by zookeeper. Otherwise, a communication error may occur. Remember to remember. You can also configure a superevisor. slot. port, supervisor. slots. ports indicates the number of worker slots of the supervisor node, that is, a maximum of several worker processes can be run (each sprout or bolt starts only one worker by default, but can be changed to multiple by using conf ).
 
All right, the storm cluster is configured.
 
Start the strom cluster:
 
Start on the master node hostA:
 
# Bin/storm nimbus
 
# Bin/storm supervisor
 
# Bin/storm ui
 
Start hostB hostC on the slave node:
 
# Bin/storm supervisor
 
 
 
Then we can see the running status of storm ui on the http: // {NimbusHost}: 8080 interface.
 
 
 
If you submit a running task:
 
Storm jar sendCloud-dataAnalysis.jar com. sohu. sendCloud. controller. SimpleTopology analysis_v1
 
So far, the entire strom cluster has been built.

1 2 Next Page