HDFs Meta Data Management

Namenode saves the file system's metadata in a different file on the local disk, with the most important two files being fsimage and edits. The fsimage contains a full snapshot of the file system metadata, and edits only contains incremental changes to the metadata.

After the Namenode is started, the fsimage is loaded directly into RAM, and the edits increment changes are introduced through playback.

The interaction between Namenode and Secondarynamenode periodically introduces edits to Fsimage:

1. Secondarynamenode boot Namenode to scroll through the edits file and start writing new content to edits.new

2. Secondarynamenode Copy the Namenode fsimage and Edtis files to the local checkpoint directory

3. Secondarynamenode loads the Fsimage file, plays back edits content, merges it into Fsimage, compresses the new Fsimage file and writes to disk

4. Secondarynamenode send the new fsimage file back to Namenode,namenode to load and apply the file directly after receiving the new Fsimage file

5. Namenode renamed Edits.new to edits

6. By default, the process is executed hourly, or it is triggered when the Namenode edits file reaches the default of 64MB
   

This article is from "Lucas" blog, please be sure to keep this source http://4292565.blog.51cto.com/4282565/1672889

HDFs Meta Data Management