Hadoop Common Errors and Handling Methods

1. hadoop-root-datanode-master.log has the following error:
ERROR org. apache. hadoop. hdfs. server. datanode. DataNode: java. io. IOException: Incompatible namespaceIDs in
As a result, datanode cannot be started.
Cause: each time namenode format is used, a new namenodeId is created, while dfs. data. the directory configured with the dir parameter contains the id created in the last format and dfs. name. the IDs in the directory configured by the dir parameter are inconsistent. Namenode format clears the data in namenode, but does not empty the data in datanode, resulting in a failure at startup. All you need to do is to clear dfs before each fotmat. data. directory of the dir parameter configuration.
Format hdfs commands

Copy codeThe Code is as follows: hadoop namenode-format

2. If datanode cannot connect to namenode, datanode cannot be started.
ERROR org. apache. hadoop. hdfs. server. datanode. DataNode: java. io. IOException: Call to... failed on local exception: java.net. NoRouteToHostException: No route to host
Disable FirewallCopy codeThe Code is as follows: service iptables stop

After the machine is restarted, the firewall is enabled.

3. the following error occurs when you upload a file from a local machine to the hdfs File System:
INFO hdfs. DFSClient: Exception in createBlockOutputStream java. io. IOException: Bad connect ack with firstBadLink
INFO hdfs. DFSClient: Abandoning block blk _-13005297058032926521337023
WARN hdfs. DFSClient: DataStreamer Exception: java. io. IOException: Unable to create new block.
Solution:
Disable Firewall:

Copy codeThe Code is as follows: service iptables stop

Disable selinux:
Edit the/etc/selinux/config file and set "SELINUX = disabled"

4. Security Mode errors
Org. apache. hadoop. dfs. SafeModeException: Cannot delete..., Name node is in safe mode
When the Distributed File System is started, there will be security mode at the beginning. When the Distributed File System is in security mode, the content in the file system cannot be modified or deleted, until the Security Mode ends. The security mode is mainly used to check the validity of data blocks on each DataNode when the system starts, and to copy or delete some data blocks as required by the policy. You can also enter security mode through commands during runtime. In practice, when the system starts, an error message indicating that the file cannot be modified in security mode is displayed when you modify or delete the file. You only need to wait for a while.

Copy codeThe Code is as follows: hadoop dfsadmin-safemode leave

Disable Security Mode