Secondarynamenode working mechanism in Hadoop

First, consider the structure of HDFS, such as:

For example, in the HDFs architecture, Namenode is responsible for managing metadata information, and Datanode's responsibility is to store the data, so what is the role of Secondarynamenode?

In fact, Secondarynamenode is a solution to the hadoop1.x in HDFs ha, let's look at the process of secondarynamenode work, such as:

1.Namenode manages the metadata information, the metadata information is periodically brushed to disk, two of which are edits is the operation log file and Fsimage is the metadata image file, the new operation log is not immediately merged with Fsimage. will not be brushed into namenode memory, but will be written to edits first (because the merge consumes a lot of resources). Checkpoint triggers Secondarynamenode to work when the size of the edits file reaches a critical value (the default is 64MB) or a period of time (by default, 1 hours).

2. When a checkpoint operation is triggered, Namenode generates a new edits edits.new file, and secondarynamenode copies the edits file and fsimage to local.

3.Secondarynamenode Loads the local fsimage file into memory and then merges with the edits file to generate a new Fsimage file in the Fsimage.ckpt file.

4.secondarynamenode Copy the newly generated fsimage.ckpt file to the Namenode node.

5. The Edits.new file and the Fsimage.ckpt file in the Namenode node replace the original edits file and the Fsimage file, so this is just a reincarnation, Namenode and edits files in Fsimage.

6. wait for the next checkpoint trigger Secondarynamenode to work, so that the loop operation has been done.

Note: Checkpoint triggered conditions can be configured in the Core-site.xml file, as follows:

[HTML]View PlainCopy

2. <property>
4. <name>fs.checkpoint.period</name>
6. <value>3600</value>
8. <description>the number of seconds between, periodic checkpoints.
10. </Description>
12. </Property>
16. <property>
18. <name>fs.checkpoint.size</name>
20. <value>67108864</value>
22. <description>the size of the current edit log (in bytes) that triggers
24. A periodic checkpoint even if the fs.checkpoint.period hasn ' t expired.
26. </Description>
28. </Property>

Secondarynamenode working mechanism in Hadoop