Startup failure caused by connection refused in zookeeper cluster construction

1. Preface

Each time the cluster environment is like a battle, the Battle of any one of the details of the error will lead to serious consequences, so the configuration needs to do the configurations such as system configuration, network configuration (firewall remember), user rights, file permissions and configuration files and so on, must be very carefully, will be killed in the cradle of mistakes, Avoid because of carelessness or want to improve speed saved a minute, in the back to pay a few hours of commissioning time, pound foolish is not worth, remember to check, check and check again.

today to say in the building zookeeper cluster encountered a common problem Connection refused, that is, many friends on the Internet will appear, the problem may be the same, but the cause of the problem is different, Therefore, we should pay attention to their log file errors, and then confirm that the external environment is not the case, focus on the resolution log errors .

2. My question

Here's what I did when I was building the cluster times:

It's a superficial mistake.
2016-12-08 05:11:07,890 [Myid:1]-WARN [quorumpeer[myid=1]/0:0:0:0:0:0:0:0:2181:[email protected]]-Cannot open Channe L to 5 at election address lzx15/192.168.80.175:3888Java.net.ConnectException:Connection refused at Java.net.PlainSocketImpl.socketConnect (Native Method) A T Java.net.AbstractPlainSocketImpl.doConnect (Abstractplainsocketimpl.java:339) at Java.net.AbstractPlainSocketImpl.connectToAddress (Abstractplainsocketimpl.java:200) at Java.net.AbstractPlainSocketImpl.connect (Abstractplainsocketimpl.java:182) at Java.net.SocksSocketImpl.connect (Sockssocketimpl.java:392) at Java.net.Socket.connect (Socket.java:579) at Org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne (Quorumcnxmanager.java:368) at Org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll (Quorumcnxmanager.java:402) at Org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader (Fastleaderelection.java:840) at Org.apache.zookeeper.server.quorum.QuorumPeer.run (Quorumpeer.java:762) at Java.net.AbstractPlainSocketImpl.connectToAddress (Abstractplainsocketimpl.java:200) at Java.net.AbstractPlainSocketImpl.connect (Abstractplainsocketimpl.java:182) at Java.net.SocksSocketImpl.connect (Sockssocketimpl.java:392) at Java.net.Socket.connect (Socket.java:579) at Org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne (Quorumcnxmanager.java:368) at Org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll (Quorumcnxmanager.java:402) at Org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader (Fastleaderelection.java:840) at Org.apache.zookeeper.server.quorum.QuorumPeer.run (Quorumpeer.java:762)

This is actually a mistake.
Java.io.FileNotFoundException:/home/hadoop/data/zookeeper/zkdata/version-2/acceptedepoch.tmp (Permissoin denied)

3. Other people'sSolution:

The main two points are as follows:

1) The firewall does not shut down causing the communication to fail , unable to connect to the other nodes, I confirm that the firewall is closed after repeated checks, so this method is not valid for my problem

Command line: Chkconfig iptables off (firewall off)

Service iptables status (view firewall status)

2) configuration file issues

A./zookeeper/conf/zoo.cfg file configuration Issues

For example, DataDir format is not correct, or do not create datadir specified files and so on, so carefully check your configuration file, as for the contents of this file please Baidu.

B. Create the myID file in the directory specified by DataDir, and then write the corresponding server number for that node in zoo.cfg, for example Server.1, then write 1 in myID.

4. My Solution

Find the log file for zookeeper Zookeeper.out where? The normal path is your Zookeper installation directory/zookeeper.out, i in this zookeeper.out can only find the surface of the error that connection refused, so has been unable to find the problem, and finally I inadvertently found

[Email protected] hadoop]$ cd/home/~]$ lsapp  data  tools  zookeeper.out

I found the real mistake of my problem in this zookeeper.out,/home/hadoop/data/zookeeper/zkdata/version-2/acceptedepoch.tmp (Permissoin denied), So I went to check the permissions of the Verison-2 folder, found that he is actually the root user, I immediately transferred it to the Hadoop user, and then restart the cluster, three nodes started successfully.

5. Summary

　　Most of the time we will go to face the surface of the problem to find solutions, but a lot of problems are only because of other problems caused by the problem, so you think to find a solution to the current problem is still not able to solve the problem, think about a place in front of the wrong, go elsewhere to find the answer, remember, The key is usually not in the lock.

Zookeeper connection refused in cluster build failed to start