Hadoop 2.x HDFs ha tutorial ten Web UI monitoring page analyze and view the edit log for NN and JN storage

So far, we've configured the HA for Hadoop, so let's go through the page to see the Hadoop file system.

1. Analyze the status of active Namenode and standby namenode for client services.

We can clearly see the directory structure of the Hadoop file system:

Above all we are accessing Hadoop through active namenode, so if we can access Hadoop through standby namenode.

Next we see that through standby namenode is inaccessible to Hadoop's file system. As prompted, we know that standby namenode is not supported for read operations.

This proves that, in the HA mechanism, standby namenode is not able to provide services to clients.

2. Analyze the Datanode of active Namenode and standby namenode.

In our HA configuration, there are a total of 3 datanode, and these 3 datanode are both active Namenode and standby namenode services.

First: We first analyze the number of Datanode under standby Namenode and where standby Namenode reads the edit log.

As we know from the above figure, there are 3 Datanode below the standby namenode, and Standby Namenode reads the edit log in Qjm way.

Second, let's take a look at how active Namenode writes the edit log to three journal node.

We can see that the current active Namenode is writing an edit log to 3 Journal node, and the current edit log's transaction ID is 41.

Thirdly, we will compare the cluster IDs of active Namenode and standby namenode, which must be the same.

Four: Comparing the Block ID, we know that the block ID stores the meta-information of the Hadoop Distributed File System because standby Namenode is the backup node of the active Namenode,

The meta information they manage should be the same, so their block ID should be the same.

Five: Now we go to the active Namenode storage directory to see.

There is a in_user.lock file that proves that the current file is being used. The current directory contains image files and edit log files that are currently stored under active Namenode.

We look at the files under the current directory:

Some people will doubt that our edit log files are not stored in journal node. Why is there also an edit log file in the active namenode?

Let's take a look at the following diagram to explain. We found that the edit log file is always available on the active Namenode side. When the client requests Namenode,

Namenode not only writes the edit log locally, but also writes the edit log file to the QJM node.

Sixth: Now we go to standby Namenode's storage directory to see.

Go to the current directory below:

We found that there were only image files and no log files were edited. Here the image file, we use a command at the time of launch, the active Namenode image file is completely copied over.

Seventh: Now we go to journal node to see the storage directory created for the current namespace (ns1).

As for why this is so configured, the official documentation has a good explanation:

We advanced to see what is stored in this directory. It should be the editing log.

The current folder is the first, then the edit log is stored under current.