- HDFS Overview and Design objectives
- What if we were to design a distributed file storage system ourselves?
- HDFs Design Goals
- A very large Distributed file system
- Running on plain, inexpensive hardware
- Easy to expand, provide users with a good performance file storage System
- HDFS Architecture
Master (NAMENODE/NN) with n slaves (DATANODE/DN)
Same as Hdfs/yarn/hbase architecture
1 files will be split into multiple blocks
blocksize:128m
130M = = "2 x block:128m and 2M
NameNode:
1. Response for client requests
2. Responsible for the management of metadata (file name, copy factor, block storage dn)
Dn:
1. Block of data corresponding to the stored user's files
2. To send the heartbeat information to the NN on a regular basis, and report all of its block information, health status
: http://archive.cloudera.com/cdh5/cdh/5/
Version number: hadoop-2.6.0-cdh5.7.0
Installation Instructions: http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0/hadoop-project-dist/hadoop-common/SingleCluster.html
Help Link: http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html
-
- Hadoop Pseudo-Installation steps
HDFs Distributed File System