Difference between fsimage and edits in hadoop

Difference between fsimage and edits in hadoop

1. concept:

Fsimage saves the latest metadata checkpoint.

Edits stores the changes in the namespace after the latest checkpoint.

2. Working principle:

After the latest checkpoint, hadoop stores operations on each file in edits. To avoid increasing edits, secondary namenode periodically merges fsimage and edits into a new fsimage, edits records new changes.

This mechanism has a problem: Because edits is stored in Namenode, when Namenode fails, edits will also be lost, resulting in the use of secondary namenode to restore Namenode, some data will be lost.