The road to mathematics-distributed computing-hdfs (2)

1, Datanode for each data copy copy, save in other Datanode.
2, Namenode file system metadata is fully insured in RAM, Datanode initialization and every 1 hours, to namenode send block report, in order to namenode grasp the operation status of each Datanode, and datanode mapping.
3, pre-write log. Reduce I/O operations by sequentially increasing in edits.
4, Fsimage:namenode start, load Fsimage to ram, introduce edits change.
5. A new version of Hadoop has been implemented to periodically replace the edits file, providing a transaction ID to identify and support data reflow.

This blog all content is original, if reproduced please indicate sourcehttp://blog.csdn.net/myhaspl/

1, Datanode for each data copy copy, save in other Datanode.

2, Namenode file system metadata is fully insured in RAM, Datanode initialization and every 1 hours, to namenode send block report, in order to namenode grasp the operation status of each Datanode, and datanode mapping.
3, pre-write log. Reduce I/O operations by sequentially increasing in edits.
4, Fsimage:namenode start, load Fsimage to ram, introduce edits change.
5. A new version of Hadoop has been implemented to periodically replace the edits file, providing a transaction ID to identify and support data reflow.
The operation log file in HDFs, the Wal, is stored in the edits.
1, Namenode will change the content to write to edits.
2, edits will slowly become larger.
3. Add edits to fsimage regularly
Namenode abbreviation nn,secondary namenode abbreviation SNN, for the second namenode.
1, secondary Namenode boot namenode rolling update edits file, the update content is written to edits.new.
2, secondary namenode copy namenode fsimage and edits files to local checkpoint directory
3. Namenode load fsimage file, merge edits content into Fsimage, compress new Fsimage file and write to disk.
4, the second namenode will send the new Fsimage file back to Namenode.
5, Namenode receive the new Fsimage file, directly loaded to make the file effective.
6, Namenode will edits.new change back to edits.
7, so far, this time to the fsimage of the completion of the update. Edits is the new content that is added later, and can continue to be used for the next iteration.

The road to mathematics-distributed computing-hdfs (2)