Experiment of NN HA for Client transparency

Previously reproduced in a "Illidan" written nn ha experimental records, the blog describes the main standby nn transparent switching process, that is, when the main nn is hung up, automatic nn switching to primary NN, Hadoop cluster normal operation.

Today, I went on to do an experiment to realize that switching over NN does not affect the client-side program, that is, nn switching is transparent to the client.

First of all, a very important point:

To ensure that the Core-site.xml

<property>    
    <name>fs.defaultFS</name>    
    <value>hdfs://hadoop-cluster</value >     
</property>

and Hdfs-site.xml in the

<property>    
    <name>dfs.nameservices</name>    
    <value>hadoop-cluster</value>    
</property>

Logical names must be consistent.

Secondly:

This column more highlights: http://www.bianceng.cnhttp://www.bianceng.cn/webkf/tools/

The/etc/hosts file does not need to be specified for Hadoop-cluster Ip,hadoop-cluster is just a logical name, and when the program or command accesses the HDFs cluster through the Hadoop-cluster, Hadoop automatically finds an active Namenode based on the Hdfs-site.xml configuration.

Of course, the hdfs-site.xml in the

<property>    
    <name>dfs.namenode.rpc-address.hadoop-cluster.namenode1</name>    
    <value >NODE001:8020</value>    
</property>    
<property>    
    <name> Dfs.namenode.rpc-address.hadoop-cluster.namenode2</name>    
    <value>NODE002:8020</value>    
</property>

The NODE001 and NODE002 in the/etc/hosts file must be configured, or Hadoop will not be smart enough to know which node to interact with.

And then there's the client.

Regardless of how namenode switches, the command Hadoop dfs-ls hdfs://hadoop-cluster/can be executed properly on any Hadoop node, or the HDFs destination address is specified as hdfs://in the program. Hadoop-cluster can run the program normally.

The key is to access HDFs not through the IP address or host name of the primary standby nn, but through the HDFs logical name.

In this case, the execution of commands and programs has nothing to do with which node the current active Namenode is, everything is transparent and it feels so good!

CSDN Blog Gandalf_lee