Hadoop installation encounters various exceptions and solutions

Last Update:2014-12-22 Source: Internet

Author: User

Keywords Java manager -13

Developer on Alibaba Coud: Build your first app with APIs, SDKs, and tutorials on the Alibaba Cloud. Read more ＞

Exception One:

2014-03-13 11:10:23,665 INFO org.apache.Hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 0 time (s); Retry policy is retryuptomaximumhttp://www.aliyun.com/zixun/aggregation/16460.html ">countwithfixedsleep ( maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:24,667 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 1 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:25,667 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 2 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:26,669 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 3 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:27,670 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 4 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:28,671 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 5 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:29,672 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 6 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:30,674 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 7 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:31,675 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 8 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:32,676 INFO org.apache.hadoop.ipc.Client:Retrying Connect to server:linux-hadoop-38/ 10.10.208.38:9000. Already tried 9 time (s); Retry policy is Retryuptomaximumcountwithfixedsleep (maxretries=10, sleeptime=1 SECONDS)

2014-03-13 11:10:32,677 WARN Org.apache.hadoop.hdfs.server.datanode.DataNode:Problem connecting to server: linux-hadoop-38/10.10.208.38:9000

Solution:

1, Ping linux-hadoop-38 can pass, Telnet linux-hadoop-38 9000 can not pass, the explanation opened the firewall

2, to linux-hadoop-38 host shutdown firewall/etc/init.d/iptables stop, show:

Iptables: Clear Firewall rules: [OK]

Iptables: Set the chain as policy accept:filter [OK]

Iptables: Uninstalling module: [OK]

3, restart

Exception Two:

2014-03-13 11:26:30,788 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:Initialization failed for block pool Block pool bp-1257313099-10.10.208.38-1394679083528 (storage ID ds-743638901-127.0.0.1-50010-1394616048958) service To linux-hadoop-38/10.10.208.38:9000

Java.io.IOException:Incompatible clusterids In/usr/local/hadoop/tmp/dfs/data:namenode Clusterid = cid-8e201022-6faa-440a-b61c-290e4ccfb006; Datanode Clusterid = clustername

At Org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition (datastorage.java:391)

At Org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead (datastorage.java:191)

At Org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead (datastorage.java:219)

At Org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage (datanode.java:916)

At Org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool (datanode.java:887)

At Org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo (bpofferservice.java:309)

At Org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake (bpserviceactor.java:218)

At Org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run (bpserviceactor.java:660)

At Java.lang.Thread.run (thread.java:662)

Solution:

1, in the Hdfs-site.xml configuration file, configured Dfs.namenode.name.dir, in master, this configuration directory has a current folder, which has a version file, which reads as follows:

#Thu 10:51:23 CST 2014

namespaceid=1615021223

clusterid=cid-8e201022-6faa-440a-b61c-290e4ccfb006

Ctime=0

Storagetype=name_node

blockpoolid=bp-1257313099-10.10.208.38-1394679083528

Layoutversion=-40

2, in the Core-site.xml configuration file, configured Hadoop.tmp.dir, in slave, the directory of this configuration has a dfs/data/current directory, which also has a version file, content

#Wed Mar 17:23:04 CST 2014

storageid=ds-414973036-10.10.208.54-50010-1394616184818

Clusterid=clustername

Ctime=0

Storagetype=data_node

Layoutversion=-40

3, at a glance, two content is not the same, caused. Delete the wrong content in the slave, reboot, fix!

References: http://www.linuxidc.com/Linux/2014-03/98598.htm

Exception Three:

2014-03-13 12:34:46,828 FATAL org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices:Failed to Initialize Mapreduce_shuffle

Java.lang.RuntimeException:No class Defiend for Mapreduce_shuffle

At Org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.init (auxservices.java:94)

At Org.apache.hadoop.yarn.service.CompositeService.init (compositeservice.java:58)

At Org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.init ( containermanagerimpl.java:181)

At Org.apache.hadoop.yarn.service.CompositeService.init (compositeservice.java:58)

At Org.apache.hadoop.yarn.server.nodemanager.NodeManager.init (nodemanager.java:185)

At Org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager (nodemanager.java:328)

At Org.apache.hadoop.yarn.server.nodemanager.NodeManager.main (nodemanager.java:351)

2014-03-13 12:34:46,830 FATAL Org.apache.hadoop.yarn.server.nodemanager.NodeManager:Error starting NodeManager