The architecture of HDFs

The introduction of the most core distributed File System HDFs, MapReduce processing, data warehousing tools hive and the distributed database HBase in the Hadoop distributed computing platform basically covers all the technical cores of the Hadoop distributed platform.

The architecture of HDFs

The entire Hadoop architecture is mainly through HDFS to achieve the underlying support for distributed storage, and through Mr to implement the Distributed Parallel task processing program support.

HDFs uses a master-slave (Master/slave) architecture model in which an HDFS cluster is composed of multiple namenode and multiple datanode (hadoop2.2 supports multiple namenode later)

Namenode acts as the primary server, managing file system namespaces and client access to files. Datanode manages the stored data. HDFS supports data in the form of files.

Internally, the file is divided into blocks of data that are stored on a set of Datanode. Namenode executes the namespace of the file system, such as opening, closing, renaming files or directories, and also responsible for mapping data blocks to specific datanode. Datanode is responsible for the file read and write of the file system client, and the creation, deletion and copying of the database under the unified dispatch of Namenode. Namenode is the manager of all HDFs metadata, and user data is never namenode.

File write:

1) The client initiates a file write request to Namenode.

2) Namenode returns information about the Datanode that the client has managed, based on file size and file block configuration.

3) The client divides the file into blocks, which, according to the Datanode address, writes the block sequentially to the Datanode block.

File read:

1) The client initiates a request to Namenode to read the file.

2) Namenode Returns the Datanode information for the file store.

3) The client reads the file information.

As a distributed file system, HDFs can be used as a reference point in data management:

File Block Placement: A block will have three backups, one on the datenode specified by Namenode, and one on Datanode with the specified Datanode not on the same machine, One is the specified datanode on the datanode on the same rack. The purpose of the backup is for data security, in order to take into account the same rack failures, as well as the performance of different data copies.

Introduction to Rack:
Http://www.cnblogs.com/ggjucheng/archive/2013/01/03/2843015.html

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

The architecture of HDFs