Big Data Note 04: HDFs for Big Data Hadoop (Distributed File System)

Source: Internet
Author: User

What is 1.HDFS?

The Hadoop Distributed File System (HDFS) is designed to be suitable for distributed file systems running on general-purpose hardware (commodity hardware). It has a lot in common with existing Distributed file systems.

Basic Concepts in 2.HDFS

(1) blocks (block)

"Block" is a fixed-size storage unit, HDFS files are partitioned into blocks for storage, HDFs block default size is 64MB. After the file is delivered, HDFs splits the file into blocks for management, and "block" is the logical unit of file storage processing.

(2) HDFs has two types of nodes: NameNode and DataNode

Namenode is the management node for HDFs, which stores file metadata.

The meta data here consists of two parts:

--->1. File and Data block mapping table

--->2. Mapping tables for data blocks and data nodes

Datenode is the working node of HDFs, which stores data blocks.

3.HDFS Architecture:

Customers who want to access the data will first send a request to the Namenode query metadata. By reading the returned results, you know which nodes the files are stored on. So to these nodes to get the database, after downloading the data block, assembled in the assembly into complete data, that is, we want to file.

Big Data Note 04: HDFs for Big Data Hadoop (Distributed File System)

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.