A small observation of the HDFS operation of Hadoop

Hadoop is now a very hot big data running framework and platform, for this amazing big guy I am not clear, the previous time to ignore it to run HADOOP, look at its operation record storage part (Operation log), IMAGE records all the platform's file operation records, such as creating files, Delete files, rename and so on, here are some of my little observations.

Formatting----Initialization

This is the initial appearance of Fsimage, because it simply formats the disk and does not have any operations. An image with a sequence number of 0, a MD5 checksum file, and a version number file.

Turn on cluster after formatting

No ACTION---1 hours later

You can see the number of IMAGE increases, the number is also increasing, of course, there is no action during this time.

After a certain operation, you can see the following two graphs: The number of images increased rapidly, the sequence number is also increasing,

The edits file records the operation record, increases over time, and the serial number increases.

Edits_inprogress is a file that is being recorded.

Fsimage is verified, at a point in time before the record of all files, which can be seen in the two storage is done separately, is the last two near the point of time.

A small observation of the HDFS operation of Hadoop