An analysis of the reading process and the writing process for the novice HDFs

Just contact HDFs, feel the data of HDFs very high reliability, record a bit.

A basic principle of HDFS

HDFs employs a master-slave (Master/slave) architecture model, and an HDFS cluster consists of a name node (NameNode) and several data nodes (DataNode). The name node is the central server that manages the namespace of the file system and the client's access to the file. The data node in the cluster is usually a node running a data node process, responsible for processing the file system client's read/write requests, the name node in the unified scheduling of data block creation, deletion and replication operations. The data for each data node is actually stored in the local Linux file system.

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M00/8A/20/wKioL1goccvQM1iDAADaWeEAxNk101.jpg-wh_500x0-wm_3 -wmp_4-s_2088887584.jpg "title=" image 2.jpg "alt=" Wkiol1goccvqm1idaadaweeaxnk101.jpg-wh_50 "/>

Two HDFs write operations

The process of writing a file: is a pipeline replication

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M02/8A/20/wKioL1gocfqS6-i3AAEaj4cGiFI558.jpg-wh_500x0-wm_3 -wmp_4-s_2158889842.jpg "title=" image 4.jpg "alt=" Wkiol1gocfqs6-i3aaeaj4cgifi558.jpg-wh_50 "/>

Three HDFs read operations

650) this.width=650; "Src=" Http://s2.51cto.com/wyfs02/M02/8A/24/wKiom1gocjmRNuosAAETak6f4eY933.jpg-wh_500x0-wm_3 -wmp_4-s_1078129482.jpg "title=" image 3.jpg "alt=" Wkiom1gocjmrnuosaaetak6f4ey933.jpg-wh_50 "/>

HDFS File Block multi-copy distributed storage mechanism to ensure the reliability of the data storage, when the file is read DataNode hung only to fail to transfer to the other copy of the DataNode continue to read, and for a copy block error, read the file data block if the checksum failure can be identified as corruption, Can still be transferred to read other intact copies, and to NameNode report the file block damage, subsequent processing by NameNode notify DataNode Delete damaged file block, and according to the good copy of a new file block copy.

This article is from the "12280240" blog, please be sure to keep this source http://12290240.blog.51cto.com/12280240/1872460

An analysis of the reading process and the writing process for the novice HDFs