Common errors in Hadoop and how to handle them

1, Hadoop-root-datanode-master.log the following error:
ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:java.io.IOException:Incompatible Namespaceids in
Cause the Datanode not to start.

Cause: Each time Namenode format re-creates a Namenodeid, the directory that the Dfs.data.dir parameter configures contains the ID of the last format creation, and the ID in the directory configured by the Dfs.name.dir parameter is inconsistent. Namenode format empties the data under Namenode, but does not empty the data under Datanode, causing the startup to fail, so the failure is due to version inconsistency.

The first of these two solutions

Enter the current directory of each datanode dfs.data.dir, and modify the file version inside

#Fri Nov 15:00:17 CST 2012
namespaceid=246015542
storageid=ds-2085496284-192.168.1.244-50010-1353654017403
Ctime=0
Storagetype=data_node
Layoutversion=-32

There's a namespaceid to change it into an error message.

Namenode Namespaceid = 971169702

The same Namespaceid.

And then restart the Datanode, all of them will start normally.

The second type:

Directly delete the files on the data node and delete all the files in the Dfs.data.dir directory above each datanode.

RM-RF *

After deletion

Reboot is also OK to start all the Datanode

all the Datanode in the Dfs.data.dir directory are regenerated. So the old data will not be recovered. In fact, Namenode formatted, the data is useless.

2. If Datanode is not connected to Namenode, Datanode cannot start.
ERROR Org.apache.hadoop.hdfs.server.datanode.DataNode:java.io.IOException:Call to ... failed on local exception: Java.net.NoRouteToHostException:No Route to host
Shutting down the firewallThe firewall will also open after the machine restarts.

3, from the local to the HDFs file system upload files, the following error occurred:
INFO HDFs. Dfsclient:exception in Createblockoutputstream Java.io.IOException:Bad connect ack with Firstbadlink

INFO HDFs. Dfsclient:abandoning Block blk_-1300529705803292651_37023 

WARN HDFs. Dfsclient:datastreamer Exception:java.io.IOException:Unable to create new block.  

Workaround:  
Shut down firewall: Span class= "Apple-converted-space" > 
Disables Selinux: 
Edit/etc/ Selinux/config file, setting "selinux=disabled"  

4, error in Safe mode  
Org.apache.hadoop.dfs.SafeModeException:Cannot Delete ..., Name node is in safe mode< Span class= "Apple-converted-space" >&NBSP;
when the Distributed file system starts, there will be a security mode at the beginning, when the distributed file system is in Safe mode, the contents of the file system are not allowed to be modified or deleted. Until Safe mode ends. The Safe mode is to check the validity of the data blocks on each datanode when the system is started, and to copy or delete some data blocks according to the policy. The run time can also be entered in safe mode through commands. In practice, when the system starts to modify and delete files will also have a safe mode does not allow the modification of the error prompt, only need to wait a while. &NBSP;

Off Safe Mode excerpt from http://www.linuxidc.com/Linux/2012-05/59749.htm

Common errors in Hadoop and how to handle them