防火牆導致的NodeManager啟動失敗

Hadoop環境CDH4.4

今天年後第一天上班（實習中），還過一個月，實習第一份實習合約就到期了~

Hadoop叢集的虛擬環境看上去太亂，所以就將3個節點（1master + 2Slaves）重啟，結果NodeManager啟動失敗。查看日誌，記錄錯誤如下：

2014-02-10 18:24:07,635 FATAL org.apache.hadoop.yarn.server.nodemanager.NodeManager: Error starting NodeManager

org.apache.hadoop.yarn.YarnException: Failed to Start org.apache.hadoop.yarn.server.nodemanager.NodeManager

at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:196)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:329)

at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:351)

Caused by: org.apache.hadoop.yarn.YarnException: Failed to Start org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl

at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.start(ContainerManagerImpl.java:248)

at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)

... 3 more

Caused by: org.apache.hadoop.yarn.YarnException: Failed to check for existence of remoteLogDir [/var/log/hadoop-yarn/apps]

at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:179)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.start(LogAggregationService.java:132)

at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)

... 5 more

2014-02-10 18:24:07,647 INFO org.apache.hadoop.ipc.Server: Stopping server on 52154

日誌顯示：無法啟動NodeManager，無法啟動ContainerManager（也就是沒有分配資源容器管理進程），也無法檢查遠程日誌目錄（在HDFS上），原因鎖定，無法與Master（具體來說是ResourceManager）通訊，然後到master上查看防火牆是否關閉，Soga，果然防火牆是開著的，由於重啟導致防火牆開啟了，然後博主將Master上的防火牆關閉，並且chkconfig iptables off進行永久關閉（重啟後不會自動開啟），再去Slave節點上啟動NodeManager，搞定！

本欄目更多精彩內容：http://www.bianceng.cnhttp://www.bianceng.cn/Network/Firewall/