Hadoop Datanode Reload failed to start resolution

The author uses a virtual machine based on a distributed Hadoop installation, because the order of shutdown Datanode and Namenode is not appropriate, so often appear datanode loading failure.

My solution is for the first time that the entire cluster has been successfully started, but the second failure due to an abnormal operation has not started properly. The first startup failure may have many causes: either due to a configuration file error write or due to an SSH password-free login configuration error.

The second reason for the error and the first start of some differences, the wrong focus should focus on the program in the operation of some dynamic loading and generated files, the author wants to discuss the second case:

Most of the reason is that the Namespaceid in the Datanode version of Hadoop is inconsistent with the Namespaceid in the version file in Namenode. And the Namespaceid of the generation of the author should be in the execution: HDFs namenode-format This command generated when.

The resolution steps are as follows:

1, first stop the related process on Namenode: switch to the/sbin directory of Hadoop:

SH stop-dfs.sh

SH stop-yarn.sh

2, switch to the corresponding/current directory of Hadoop to clear all files under current.

3, the Datanode and Namenode/current version of the corresponding file files cleared, back to Namenode, execute HSFS namenode-format command, and then switch to Namenode of Hadoop's/ Sbin directory:

Perform SH start-dfs.sh

SH start-yarn.sh

(The old version of Mapre is replaced by a new version of the yarn, and the commands are somewhat different.)

You can see that the corresponding node is loaded successfully.

The corresponding thought is, when the error, clear away all interference ideas of the document, and then tidy up the thoughts, start again, so far more than in situ wandering better.

(Because the folder we specify in the configuration file has only HDFs tmp log, the rest of the files and folders are dynamically executed by the script generation, and as long as the entire system of Hadoop can work, it will be generated, even if the error is deleted, the VM's snapshot will save the world.) )