Introduce new DataNode nodes in the Hadoop Cluster

Source: Internet
Author: User
For example, if the ip address of the newly added node is, add the hosts of 192.168.1.xxxdatanode-xxx to all nn and dn nodes, create useraddhadoop-sbinbash-m on xxx, and add the ip address of another dn. all files in ssh are copied to homehadoop on xxx. install jdkapt-getinstallsun-java6-j in ssh path

For example, if the ip address of the newly added node is, add the hosts of datanode-xxx to all nn and dn nodes, create useradd hadoop-s/bin/bash-m on xxx, and add other dn. copy all files in ssh to/home/hadoop/on xxx /. install jdk apt-get install sun-java6-j In the ssh path

For example, the ip address of the newly added node is
Add the hosts of datanode-xxx to all nn and dn nodes.

Create a user on xxx
Useradd hadoop-s/bin/bash-m
Copy all. ssh files from other dn instances to the/home/hadoop/. ssh path on xxx.

Install jdk
Apt-get install sun-java6-jdk
Copy the hadoop file to/home/hadoop

Modify slaves files
Add datanode-xxx
And synchronized to various cluster nodes

Switch to hadoop user
Start a Data Node
Bin/ start datanode
Bin/ start tasktracker

Data Synchronization will be performed after completion.
Bin/ 3

For new nodes:
You need to reset the hostname and the hostname to the same content as the slaves configuration file.
Hostname datanode-xxx (expired after restart)
Modify/etc/hostname and run/etc/init. d/hostname. sh start (it will not expire after restart)

Then you can use
Http: // namenodeip: 50070 to check whether the newly added node can be normally introduced to the cluster.

Original article address: Introduce a new DataNode node to the Hadoop cluster. Thank you for sharing your feedback.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.