Hadoop Datanode Reload failed to start phenomenon workaround introduction

My workaround is for the first time that the entire cluster has been successfully started, but the second failure to start normally due to an unhealthy operation. There may be many reasons for the first startup failure: either due to misconfiguration of the configuration file or due to an SSH login configuration error without a password

The author uses a virtual machine based Hadoop distributed installation, because the order of shutting down Datanode and Namenode is inappropriate, so datanode load failure often occurs.

My workaround is for the first time that the entire cluster has been successfully started, but the second failure to start normally due to an unhealthy operation. There may be many reasons for the first startup failure: either due to misconfiguration of the configuration file, or due to an SSH password-free login configuration error.

The second cause of the error and the first start of some differences, the wrong focus should focus on the program in the running of some dynamic loading of the files generated, I would like to discuss the second situation:

Most of the reason is that the Namespaceid in the Datanode version file of Hadoop is inconsistent with the Namespaceid in the version file in Namenode. The author of Namespaceid's generation infers that it should be generated at the time of execution: HDFs namenode-format this command.

The steps to resolve are as follows:

1, first stop the related process on Namenode: switch to the/sbin directory of Hadoop:

SH stop-dfs.sh

SH stop-yarn.sh

2. Switch to the corresponding/current directory in Hadoop to erase all files under current.

3, the Datanode and Namenode/current under version and other corresponding file files purged, back to Namenode, execute HSFS namenode-format command, and then switch to the namenode of Hadoop/ Under the Sbin directory:

Execute SH start-dfs.sh

SH start-yarn.sh

(The old version of Mapre is replaced by the new version of yarn, the command is somewhat different)

You can see that the corresponding node is loaded successfully.

The corresponding idea is, when the error, clear out all the interference ideas of the document, and then organize your thoughts, start again, so far than in situ wandering better.

(Because the folder we specify in the configuration file is only HDFs tmp log, so the rest of the files and folders are dynamically executed script generation created, deleted as long as the entire system of Hadoop can work will be generated, even if mistakenly deleted, the VM's snapshot will save the world. ）