Zookeeper+jstorm's Cluster construction

Zookeeper configuration: Zookeeper has three configurations: stand-alone/pseudo-distributed/clustered where pseudo-distribution simulates distributed scenarios on a single computer through different ports, requires n-copies of configuration files and startup programs, and clusters are multiple zookeeper programs. First get the tar package through SFTP, extract to the specified directory:/usr/local/java, generate directory zookeeper-3.3.6; Configure environment variables (note that there is no space after the path, there is no space between the path and the equal sign.) )：

Vi/etc/profile #zkZOOKEEPER_HOME =/usr/local/java/zookeeper-3.3.6export path= "$PATH: $ZOOKEEPER _home/bin" Export Zookeeper_home

Cd/zookeeper-3.3.6/conftouch zoo.cfg or MV zoo-sample.cfg Zoo.cfgvi zoo.cfg Insert the following: ticktime=2000 InitLimit=10syncLimit =5datadir=/usr/local/java/zookeeper-3.3.6/datadatalogdir=/usr/local/java/zookeeper-3.3.6/logsclientport= 4180server.3=192.168.204.3:2888:3888server.4=192.168.204.4:2888:3888server.5=192.168.204.5:2888:3888-- Where the number of nodes is greater than the number of virtual machines, the file path must exist, there is no subsequent to create, there can be no space after the path

Complete configuration: ESC, WQCD. To the root directory mkdir Logsmkdir data needs to be set up under Data a myid asking price, content is the server number, must be native server number touch Myidvi myid Insert host corresponding server number

The basic configuration is this, and then we perform the same configuration on the other two VMs, Zoo.cfg basically the same, but the myID is based on the server number of the native IP map. The same path can not have a space to start three virtual machine zookeeper,zkserver.sh start Remember when the first start, will error can not find the back of the two, this does not matter, it is OK to start. via the command below the bin: zkserver.sh status to see if a node is leader or follower after successful startup, indicating that the boot was successful. In the middle encountered some problems, mainly can not start, after startup can not communicate, find files and so on. The reasons are: myID lost, Bin is not configured to path, the configuration of the logs folder is not established, zoo.cfg path has spaces, firewall blocking and so on. It had already run successfully, but the next day came, and then there was a problem.performance is:zhserver.sh start starts, the started is displayed, and the background process is running, and the ZK is started in turn. The JPS command can also see the quorumpeermain process. Obviously, it's already started.But zkserver.sh status command to view, but also error:Error contacting service. It is probably not running. see a lot of information ah, what change hosts, check myID, check whether the folder is set up, but not targeted to change, always no results. And the iptables command also shows invalid (I thought it was closed) Summary: The beginning of the error, and did not seriously read the wrong log, mainly online blog said when the error is normal, because the other nodes did not start. So there is no further depth, and then all kinds of aimless blind touch ...

Error following java.net.NoRouteToHostException:No route to host (no route to host) after checking the network, port, Hosts,iptables is not used. Then found Centos7 's firewall is no longer iptables, become firewall. I didn't use the service iptables Stop command before. Last Pass:--------------------------------------------------------------centos7.0 (the default is to use firewall as the firewall, If you have not changed to iptables firewall, use the following command to view and close the firewall) view firewall status: Firewall-cmd--state shut down firewall: Systemctl stop Firewalld.service------------------------------------------------------------------Ubuntu (ubuntu-12.04-desktop-amd64) View Fire protection Wall Status: UFW status shutdown Firewall: UFW Disable---------------------------------------------------------------CentOS6.0 View firewall status: Service iptables Status shutdown firewall: chkconfig iptables off    #开机不启动防火墙服务

Shut down the firewall, so no error. It's easy to set up, but it's inefficient to linger for a long time. (Baidu actually also said the status command has a problem)In addition, in order to prevent the next time to disrupt, select Disable boot: systemctl disable Firewalld.service after the command has been knocked out:removed Symlink/etc/systemd/system/dbus-org.fedoraproject.firewalld1.service.Removed symlink/etc/systemd/system/basic.target.wants/firewalld.service. This step should be removed from the boot-up add-on, implementing a no-boot boot. If you need it later, you can copy it in again. Reference: "Http://www.cnblogs.com/silent2012/archive/2015/07/28/4682770.html" So, starting zkserver.sh start in turn shows that the start is successful, Then zkserver.sh status shows:

[Email protected] bin]# zkserver.sh statusjmx enabled by defaultusing Config:/usr/local/java/zookeeper-3.3.6/bin/. /conf/zoo.cfgmode:follower[[email protected] bin]# jps2473 QuorumPeerMain2661 Jps

That is success. Remember that the error must be around zookeeper.out. Other questions:1. Terminal is always not connected to one of the virtual machine, I think where the configuration is not correct, delete and re-clone, reconfigure, but still no, later changed a terminal program is good, really day the dog's mood. whether it's a command line with a UI or a virtual machine without a UI, it's hard to use. Only through the Xshell tool to connect, can be very convenient to edit. Jstorm Configuring Jstorm Environment variables:

Vi/etc/profile inserting #jstormexport jstorm_home=/usr/local/java/jstorm-0.9.6export path= $PATH: $JSTORM _home/bin

Jstorm configuration file: can refer to the server on the configured files, combined with their own cluster, make some changes can be.

########### these must be filled on for a storm configuration storm.zookeeper.servers:-"192.168.204.3"-"192.16 8.204.4 "-" 192.168.204.5 "Storm.zookeeper.root:"/jstorm "Nimbus.host:" 192.168.204.3 "#%jstorm_home% is the Jstorm Home directory Storm.local.dir: "%jstorm_home%/data" Java.library.path: "/usr/local/java/jstorm-0.9.6/lib:/usr/ Local/lib:/opt/local/lib:/usr/lib "Supervisor.slots.ports:-6800-6801-6802-6803# Default Disable user-d Efine classloader# If There is jar conflict between Jstorm and application, # Please enable it Topology.enable.classload er:false# Enable supervisor use Cgroup to make resource isolation# before enable it, you should make sure:# 1. Linux version (>= 2.6.18) # 2. There are installed Cgroup (check the file ' s existence:/proc/cgroups) #3. You should start your supervisor on root# you can get more about cgroup:# Http://t.cn/8s7nexU Supervisor.enable.cgroup: false### Netty'll send multiple messages in one batch # # # Setting true would improve throughput, but more latency Storm.messaging.netty.transfer.async.batch:true # # # # Default worker Mem Ory size, Unit is byte worker.memory.size:2147483648# Metrics monitor# topology.performance.metrics:it is the switch FLA G for performance # purpose. When it was disabled, the data of timer and histogram metrics # would not be collected.# topology.alimonitor.metrics.post:i F It is disable, metrics data# would are only being printed to log. If It is enabled, the metrics data would be# posted to alimonitor besides printing to log.         Topology.performance.metrics:true topology.alimonitor.metrics.post:falseui.clusters:-{name: "Jstorm",  Zkroot: "/jstorm", zkservers: ["192.168.204.3", "192.168.204.4", "192.168.204.5"], Zkport: 2181,}

Here we need to download the official example run on the UI, do the test, but git can not support only clone which example directory, this normal requirement. It's disappointing. View port informationNetstat-ntulp |grep 

 ui.clusters: 
  -{
  name:   "Jstorm", 
   Zkroot:   "/jstorm", 
  zkservers: 
  ["192.168.204.3", "192.168.204.4", "192.168.204.5"], 
  zkport:4180    } 
   Topology.backpressure.enable:  true  

6800
1.2
# supervisor.slots.port.cpu.weight:0.5
0.7
# supervisor.slots.port.mem.weight:0.5
Null

 http://blog.csdn.net/morning99/article/details/40426133http://my.oschina.net/bigsloth/blog/387297http:// blog.csdn.net/zzu09huixu/article/details/26452947http://coolxing.iteye.com/blog/1871009

From for notes (Wiz)

List of attachments

Zookeeper+jstorm's Cluster construction