HDFs Distributed File System

• HDFS Overview and Design objectives
  • What if we were to design a distributed file storage system ourselves?
  • HDFs Design Goals
    • A very large Distributed file system
    • Running on plain, inexpensive hardware
    • Easy to expand, provide users with a good performance file storage System
• HDFS Architecture

Master (NAMENODE/NN) with n slaves (DATANODE/DN)

Same as Hdfs/yarn/hbase architecture

1 files will be split into multiple blocks

blocksize:128m

130M = = "2 x block:128m and 2M

NameNode:

1. Response for client requests

2. Responsible for the management of metadata (file name, copy factor, block storage dn)

Dn:

1. Block of data corresponding to the stored user's files

2. To send the heartbeat information to the NN on a regular basis, and report all of its block information, health status

　　

• HDFs replica mechanism

• • HDFS Copy Storage Policy

• HDFS Environment Setup

: http://archive.cloudera.com/cdh5/cdh/5/

Version number: hadoop-2.6.0-cdh5.7.0

Installation Instructions: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0/hadoop-project-dist/hadoop-common/SingleCluster.html

Help Link: http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html

• • Hadoop Pseudo-Installation steps

HDFs Distributed File System