Fsimage and edits log editing logs in Hadoop

Reprint please specify the Source: "Http://datasearch.ruc.edu.cn/~boliangfeng/blog", thank you.

See this question in the Hadoopor forum, here's an answer.

I have a doubt that the Fsimsage information is recorded in Namenode's memory,
But the in-memory fsimage metadata is merged locally at namemode startup.
Editlog and Fsimage, so there are the following problems:
1. If Namenode has not been restarted,
So how do you make sure that the fsimage in memory is up to date?
2. Whether to merge local editlog and fsimage information periodically in the latest HDFs version.
3. When the metadata in the local fsimage is written.
Log is logged to Editlog when you create a new file
is the metadata written to fsimage at this time?
If so, then why Editlog to merge with Fsimage.
Because the metadata recorded in the Editlog is already in the fsimage.
It would be appreciated if you could help me with the solution.

Answer:
1: The problem itself is problematic, fsimage is a meta-data checkpoint stored on the hard disk, Hadoop does not write to fsimage for each file operation, so it is very slow, but each file operation will be written edits edit log before the Commit, This allows the fsimage and edits edit logs to be read into memory and the metadata information rebuilt after the Namenode fails. Specific checkpoints and log scrolling can be found in the database checkpoint and Apache log scrolling technology.

2: Above also said, reference database of the checkpoint knowledge. In order to avoid unlimited expansion of the edits edit log, secondary namenode back to the time threshold (e.g. 1 hours) or by the size threshold (edits edit log file size exceeds 64M, these parameters can be set) "periodicity" Edits and fsimage refactoring fsimage checkpoints in the read Namenode, while Namenode scrolling in edits to solve this problem.

3: Correctly understand the Namenode and secondary namenode mechanism to understand that the fsimage is "cyclical" to be updated.
Z