Datanode cannot connect to the master

For the first time, hadoop was configured on the VM, and three virtual machines were created, one as namenode and jobtracker.

The other two machines are used as datanode and tasktracker.

After configuration, start the Cluster

View Cluster status through http: // localhost: 50700

Hadoop configuration datanode cannot connect to the master

No datanode found

Check the node and find that the datanode process has been started. view the logs on the datanode machine.

 

2014-03-01 22:11:17,473 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-03-01 22:11:18,477 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-03-01 22:11:19,481 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-03-01 22:11:20,485 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-03-01 22:11:21,489 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: Master/10.207.160.34:9000. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

It is found that datanode cannot connect to the master node. However, after trying, you can ping it to the node to check that port 9000 is also in the listening status.

Eventually found in core-site.xml

<property><name>fs.default.name</name><value>hdfs://localhost:9000</value></property> 

 

The listening port 127.0.0.1 cannot be accessed by external servers.

Change to the host name. Everything works.

<property><name>fs.default.name</name><value>hdfs://Master:9000</value></property>

Hosts:

10.207.160.34 Master

Don't think this is the case. In fact, this is a pitfall. Don't believe it. Try it:

[email protected]_160_34_centos:~>  netstat -an | grep 9000tcp        0      0 127.0.0.1:9000              0.0.0.0:*                   LISTEN      tcp        0      0 127.0.0.1:9000              127.0.0.1:45889             ESTABLISHED tcp        0      0 127.0.0.1:45889             127.0.0.1:9000              ESTABLISHED 

Write the IP address directly.

<value>hdfs://10.207.160.34:9000</value> 

Just now.

[email protected]_160_34_centos:/usr/local/hadoop-2.4.0> netstat -nlpa|grep 9000(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)tcp        0      0 10.207.160.34:9000          0.0.0.0:*                   LISTEN      5753/java           tcp        0      0 10.207.160.34:36093         10.207.160.34:9000          TIME_WAIT   -                   tcp        0      0 10.207.160.34:9000          10.221.32.234:44229         ESTABLISHED 5753/java 

Let's report again:

[email protected]_160_34_centos:/usr/local/hadoop-2.4.0> hadoop dfsadmin -reportDEPRECATED: Use of this script to execute hdfs command is deprecated.Instead use the hdfs command for it.Configured Capacity: 8452341760 (7.87 GB)Present Capacity: 4833366016 (4.50 GB)DFS Remaining: 4833341440 (4.50 GB)DFS Used: 24576 (24 KB)DFS Used%: 0.00%Under replicated blocks: 0Blocks with corrupt replicas: 0Missing blocks: 0-------------------------------------------------Datanodes available: 1 (1 total, 0 dead)Live datanodes:Name: 10.221.32.234:50010 (Node1)Hostname: TENCENT64.siteDecommission Status : NormalConfigured Capacity: 8452341760 (7.87 GB)DFS Used: 24576 (24 KB)Non DFS Used: 3618975744 (3.37 GB)DFS Remaining: 4833341440 (4.50 GB)DFS Used%: 0.00%DFS Remaining%: 57.18%Configured Cache Capacity: 0 (0 B)Cache Used: 0 (0 B)Cache Remaining: 0 (0 B)Cache Used%: 100.00%Cache Remaining%: 0.00%Last contact: Fri Aug 29 15:39:54 CST 2014

 

Datanode cannot connect to the master