namenode and datanode in hadoop

Background information: Recently, a lot of datanode node disk IO is relatively high, the main reason is due to the increase in job number, as well as the size of the increase.But any way to reduce disk IO consumption, we can all try. For example, we can often see HDFs users executing the "du-sk" command: code is as follows copy code [Root@idc1-serv Er2 ~]# ps-ef| grep "Du-sk" hdfs 17119 10336 1 00:57? 00:00:04 d

After the Hadoop namenode reformat, restart HBase, find its hmaster process to disappear immediately after the start, look at a lot of logs, and finally found the following problems in the zookeeper logUnable to read additional data from client SessionID 0x14e86607c850007, likely client have closed socketWorkaround: Delete the hdfs-site.xml of hbase in the directory under the configured path of the content,

In the previous post there is a description of dynamically adding a node, this time explains how to dynamically delete a node.In the previous post there is a tutorial on how to limit the connection of a node, to dynamically delete a node, you can configure on this basis.1. Configure the Dfs.hosts.exclude on the host1Add Host4 to the file specified by Dfs.hosts.exclude:Then execute the following command:hadoop dfsadmin -refreshNodesThen use the following command to view:hadoop dfsadmin -repor