Hadoop Learning Notes (2) Hadoop framework parsing

Source: Internet
Author: User

Hadoop is a distributed storage and computing platform for Big data

Architecture of HDFs: Master-Slave architecture

The primary node has only one namenode, and there can be many datanode from the node.

Namenode is responsible for:

(1) Receiving User action request

(2) Maintaining the directory structure of the file system

(3) Managing the relationship between the file and block, and the connection between block and Datanode

Datanode is responsible for:

(1) Storing files

(2) file is partitioned into blocks and stored on disk

(3) To ensure data security, the file will have multiple copies

Namenode and Datanode refer to different independent physical machines.

Analogy: Block Puzzle, Namenode is a manual, each block is datanode.

MapReduce Architecture: Master-Slave architecture

The primary node has only one jobtracker, and there can be many tasktracker from the node.

Jobtracker is responsible for:

(1) Receiving the calculation task submitted by the customer

(2) Assign the calculation task to Tasktracker execution

(3) Monitoring the implementation of Tasktracker

Tasktracer is responsible for:

(1) Perform calculation tasks for Jobtracer assignment

The physical cluster distribution of Hadoop:

Each of these nodes, whether primary or slave, is essentially a Java process.

Physical structure of a single node:

So the features of Hadoop:

(1) Distributed: Strong capacity, low cost, high efficiency

(2) Replica mechanism: High reliability

Hadoop Learning Notes (2) Hadoop framework parsing

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.