Various exceptions during Hadoop installation and Solutions

Exception 1:

2014-03-13 11:10:23, 665 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 0 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)

2014-03-13 11:10:24, 667 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 1 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:25, 667 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 2 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:26, 669 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 3 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:27, 670 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 4 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:28, 671 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 5 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:29, 672 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 6 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:30, 674 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 7 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:31, 675 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 8 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
2014-03-13 11:10:32, 676 INFO org. apache. hadoop. ipc. client: Retrying connect to server: Linux-hadoop-38/10.10.208.38: 9000. already tried 9 time (s); retry policy is RetryUpToMaximumCountWithFixedSleep (maxRetries = 10, sleepTime = 1 SECONDS)
11:10:32, 677 WARN org. apache. hadoop. hdfs. server. datanode. DataNode: Problem connecting to server: Linux-hadoop-38/10.10.208.38: 9000
Solution:
1, ping Linux-hadoop-38 can pass, telnet Linux-hadoop-38 9000 can not pass, indicating that the firewall is enabled
2, go to the Linux-hadoop-38 host to close the Firewall/etc/init. d/iptables stop, display:
Iptables: clear firewall rules: [OK]
Iptables: Set the chain to policy ACCEPT: filter [OK]
Iptables: uninstalling module: [OK]

3. Restart

Exception 2:

11:26:30, 788 FATAL org. apache. hadoop. hdfs. server. datanode. DataNode: Initialization failed for block pool BP-1257313099-10.10.208.38-1394679083528 (storage id DS-743638901-127.0.0.1-50010-1394616048958) service to Linux-hadoop-38/10.10.208.38: 9000
Java. io. IOException: Incompatible clusterIDs in/usr/local/hadoop/tmp/dfs/data: namenode clusterID = CID-8e201022-6faa-440a-b61c-290e4ccfb006; datanode clusterID = clustername
At org. apache. hadoop. hdfs. server. datanode. DataStorage. doTransition (DataStorage. java: 391)
At org. apache. hadoop. hdfs. server. datanode. DataStorage. recoverTransitionRead (DataStorage. java: 191)
At org. apache. hadoop. hdfs. server. datanode. DataStorage. recoverTransitionRead (DataStorage. java: 219)
At org. apache. hadoop. hdfs. server. datanode. DataNode. initStorage (DataNode. java: 916)
At org. apache. hadoop. hdfs. server. datanode. DataNode. initBlockPool (DataNode. java: 887)
At org. apache. hadoop. hdfs. server. datanode. BPOfferService. verifyAndSetNamespaceInfo (BPOfferService. java: 309)
At org. apache. hadoop. hdfs. server. datanode. BPServiceActor. connectToNNAndHandshake (BPServiceActor. java: 218)
At org. apache. hadoop. hdfs. server. datanode. BPServiceActor. run (BPServiceActor. java: 660)
At java. lang. Thread. run (Thread. java: 662)
Solution:
1. the dfs is configured in the hdfs-site.xml configuration file. namenode. name. dir, In the master, there is a current folder under the configuration Directory, which has a VERSION file, the content is as follows:
# Thu Mar 13 10:51:23 CST 2014
Namespace id = 1615021223
ClusterID = CID-8e201022-6faa-440a-b61c-290e4ccfb006
CTime = 0
StorageType = NAME_NODE
BlockpoolID = BP-1257313099-10.10.208.38-1394679083528
LayoutVersion =-40
2. hadoop is configured in the core-site.xml configuration file. tmp. dir, in slave, the directory under this configuration has a dfs/data/current Directory, which also contains a VERSION file, content
# Wed Mar 12 17:23:04 CST 2014
StorageID = DS-414973036-10.10.208.54-50010-1394616184818
ClusterID = clustername
CTime = 0
StorageType = DATA_NODE
LayoutVersion =-40
3. At a glance, the two contents are different, resulting in. Delete the error content in slave, restart, and fix it!

References:

Exception 3:

12:34:46, 828 FATAL org. apache. hadoop. yarn. server. nodemanager. containermanager. AuxServices: Failed to initialize mapreduce_shuffle
Java. lang. RuntimeException: No class defiend for mapreduce_shuffle
At org. apache. hadoop. yarn. server. nodemanager. containermanager. AuxServices. init (AuxServices. java: 94)
At org. apache. hadoop. yarn. service. CompositeService. init (CompositeService. java: 58)
At org. apache. hadoop. yarn. server. nodemanager. containermanager. ContainerManagerImpl. init (ContainerManagerImpl. java: 181)
At org. apache. hadoop. yarn. service. CompositeService. init (CompositeService. java: 58)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. init (NodeManager. java: 185)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. initAndStartNodeManager (NodeManager. java: 328)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. main (NodeManager. java: 351)
12:34:46, 830 FATAL org. apache. hadoop. yarn. server. nodemanager. NodeManager: Error starting NodeManager
Java. lang. RuntimeException: No class defiend for mapreduce_shuffle
At org. apache. hadoop. yarn. server. nodemanager. containermanager. AuxServices. init (AuxServices. java: 94)
At org. apache. hadoop. yarn. service. CompositeService. init (CompositeService. java: 58)
At org. apache. hadoop. yarn. server. nodemanager. containermanager. ContainerManagerImpl. init (ContainerManagerImpl. java: 181)
At org. apache. hadoop. yarn. service. CompositeService. init (CompositeService. java: 58)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. init (NodeManager. java: 185)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. initAndStartNodeManager (NodeManager. java: 328)
At org. apache. hadoop. yarn. server. nodemanager. NodeManager. main (NodeManager. java: 351)
2014-03-13 12:34:46, 846 INFO org. apache. hadoop. yarn. server. nodemanager. containermanager. monitor. containersMonitorImpl: ResourceCalculatorPlugin is unavailable on this system. org. apache. hadoop. yarn. server. nodemanager. containermanager. monitor. containersMonitorImpl is disabled.
Solution:
1, yarn-site.xml configuration error:
<Property>
<Name> yarn. nodemanager. aux-services </name>
<Value> mapreduce_shuffle </value>
</Property>
2. Change:
<Property>
<Name> yarn. nodemanager. aux-services </name>
<Value> mapreduce. shuffle </value>
</Property>
3. Restart the service

 

Warning:
WARN util. NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Solution:

Reference

Exception 4:

14/03/13 17:25:41 ERROR lzo. GPLNativeCodeLoader: cocould not load native gpl library
Java. lang. UnsatisfiedLinkError: no gplcompression in java. library. path
At java. lang. ClassLoader. loadLibrary (ClassLoader. java: 1734)
At java. lang. Runtime. loadLibrary0 (Runtime. java: 823)
At java. lang. System. loadLibrary (System. java: 1028)
At com. hadoop. compression. lzo. GPLNativeCodeLoader. <clinit> (GPLNativeCodeLoader. java: 32)
At com. hadoop. compression. lzo. LzoCodec. <clinit> (LzoCodec. java: 67)
At com. hadoop. compression. lzo. LzoIndexer. <init> (LzoIndexer. java: 36)
At com. hadoop. compression. lzo. LzoIndexer. main (LzoIndexer. java: 134)
At sun. reflect. NativeMethodAccessorImpl. invoke0 (Native Method)
At sun. reflect. NativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 39)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 25)
At java. lang. reflect. Method. invoke (Method. java: 597)
At org. apache. hadoop. util. RunJar. main (RunJar. java: 208)
14/03/13 17:25:41 ERROR lzo. LzoCodec: Cannot load native-lzo without native-hadoop
14/03/13 17:25:43 INFO lzo. LzoIndexer: [INDEX] LZO Indexing file/test2.lzo, size 0.00 GB...
Exception in thread "main" java. lang. RuntimeException: native-lzo library not available
At com. hadoop. compression. lzo. LzopCodec. createDecompressor (LzopCodec. java: 91)
At com. hadoop. compression. lzo. LzoIndex. createIndex (LzoIndex. java: 222)
At com. hadoop. compression. lzo. LzoIndexer. indexSingleFile (LzoIndexer. java: 117)
At com. hadoop. compression. lzo. LzoIndexer. indexInternal (LzoIndexer. java: 98)
At com. hadoop. compression. lzo. LzoIndexer. index (LzoIndexer. java: 52)
At com. hadoop. compression. lzo. LzoIndexer. main (LzoIndexer. java: 137)
At sun. reflect. NativeMethodAccessorImpl. invoke0 (Native Method)
At sun. reflect. NativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 39)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 25)
At java. lang. reflect. Method. invoke (Method. java: 597)
At org. apache. hadoop. util. RunJar. main (RunJar. java: 208)

Solution: Obviously, there is no native-lzo
Compile and install/compile lzo,

 

Exception 5:

14/03/17 10:23:59 INFO mapreduce. JobSubmitter: Cleaning up the staging area/tmp/hadoop-yarn/staging/hadoop/. staging/job_1394702706596_0003
Java. lang. IllegalArgumentException: Compression codec com. hadoop. compression. lzo. LzoCodec not found.
At org. apache. hadoop. io. compress. CompressionCodecFactory. getCodecClasses (CompressionCodecFactory. java: 134)
At org. apache. hadoop. io. compress. CompressionCodecFactory. <init> (CompressionCodecFactory. java: 174)
At org. apache. hadoop. mapreduce. lib. input. TextInputFormat. isSplitable (TextInputFormat. java: 58)
At org. apache. hadoop. mapreduce. lib. input. FileInputFormat. getSplits (FileInputFormat. java: 276)
At org. apache. hadoop. mapreduce. JobSubmitter. writeNewSplits (JobSubmitter. java: 468)
At org. apache. hadoop. mapreduce. JobSubmitter. writeSplits (JobSubmitter. java: 485)
At org. apache. hadoop. mapreduce. JobSubmitter. submitJobInternal (JobSubmitter. java: 369)
At org. apache. hadoop. mapreduce. Job $ 11.run( Job. java: 1269)
At org. apache. hadoop. mapreduce. Job $ 11.run( Job. java: 1266)
At java. security. AccessController. doPrivileged (Native Method)
At javax. security. auth. Subject. doAs (Subject. java: 396)
At org. apache. hadoop. security. UserGroupInformation. doAs (UserGroupInformation. java: 1408)
At org. apache. hadoop. mapreduce. Job. submit (Job. java: 1266)
At org. apache. hadoop. mapreduce. Job. waitForCompletion (Job. java: 1287)
At org. apache. hadoop. examples. WordCount. main (WordCount. java: 84)
At sun. reflect. NativeMethodAccessorImpl. invoke0 (Native Method)
At sun. reflect. NativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 39)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 25)
At java. lang. reflect. Method. invoke (Method. java: 597)
At org. apache. hadoop. util. ProgramDriver $ ProgramDescription. invoke (ProgramDriver. java: 72)
At org. apache. hadoop. util. ProgramDriver. driver (ProgramDriver. java: 144)
At org. apache. hadoop. examples. ExampleDriver. main (ExampleDriver. java: 68)
At sun. reflect. NativeMethodAccessorImpl. invoke0 (Native Method)
At sun. reflect. NativeMethodAccessorImpl. invoke (NativeMethodAccessorImpl. java: 39)
At sun. reflect. DelegatingMethodAccessorImpl. invoke (DelegatingMethodAccessorImpl. java: 25)
At java. lang. reflect. Method. invoke (Method. java: 597)
At org. apache. hadoop. util. RunJar. main (RunJar. java: 208)
Caused by: java. lang. ClassNotFoundException: Class com. hadoop. compression. lzo. LzoCodec not found
At org. apache. hadoop. conf. Configuration. getClassByName (Configuration. java: 1680)
At org. apache. hadoop. io. compress. CompressionCodecFactory. getCodecClasses (CompressionCodecFactory. java: 127)
... 26 more
Temporary solution:
Copy/usr/local/hadoop/lib/hadoop-lzo-0.4.10.jar to/usr/local/jdk/lib and restart linux

 

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

1 2 Next Page