Difference between fsimage and edits in hadoop, hadoopfsimage

Difference between fsimage and edits in hadoop, hadoopfsimage

1. concept:

Fsimage saves the latest metadata checkpoint.

Edits stores the changes in the namespace after the latest checkpoint.

2. Working principle:

After the latest checkpoint, hadoop stores operations on each file in edits. To avoid increasing edits, secondary namenode periodically merges fsimage and edits into a new fsimage, edits records new changes.

This mechanism has a problem: Because edits is stored in Namenode, when Namenode fails, edits will also be lost, resulting in the use of secondary namenode to restore Namenode, some data will be lost.

Merging fsimage and edits in namenode of hadoop and hdfs

This does not affect the reading of files. During reading, nn will read the fsimage and edits files to find the file information.

What is the format operation for NameNode?

On the NameNode node, there are two most important paths used to store metadata information and operation logs. When-format is executed, will the real data be deleted? View Original post>

Hope to adopt