HDFs Fundamentals (Hadoop1.0)

One, the client sends a write block request

Principle Introduction:

1, the client sends a write block request to the Namenode, NodeName receives this request to the client a message, that is the client should write the data to which Datanode, the client then writes the data to the assigned Datanode.

2, at this time, regardless of whether the client writes the data is successful, is written in the edits journal, at this time the number +1 in edits, if the write is successful, the corresponding metadata information will be added in memory (MetaData)

Second, the renewal principle of Namenode

Principle Introduction:

1, the trigger of Namenode update mechanism. There are two conditions: one is time, the maximum time interval is 3,600 seconds, the second is the size of the edits file, the default is when the edits file is greater than 64M, and regardless of whether the maximum time interval has been reached, the fsimage (that is, some metadata information) is forced to be updated;

2, when the Namenode update mechanism is triggered, this time will enter a switching state, that is, if you receive another user's write request, the operation information will not be placed in the edits, but into the file Edits.new file, which is to keep the data consistent.

3, Secondnamenode will download two files from Namenode, edits and Fsimage, through which two files will be synthesized a new fsimage (this is called fsiamge.ckpt).

4, when edits and fsimage information Update, will send the latest fsimage.ckpt to Namenode;

5, at this time Fsimage.ckpt will replace the original Fsimage file, at this time will be deleted edits file, with Edits.new file replacement, the previous reason not to delete the edits file is because when the Secondnamenode update fails, you can use again.

HDFs Fundamentals (Hadoop1.0)