Deployment of Hadoop2.2.0
Machine environment:
Operating System: CentOS 6.4 64-bit System
Hadoop version: hadoop-2.2.0, self-compiled 64-bit version under CentOS.
Procedure:
1. Assume there are four machines in total, and the username of each machine is set to myhadoop (for ease of installation and configuration, and for permission issues)
Machine name IP Address Allocation
Hadoop1 10.172.169.191 namenode, ResourceManager
Hadoop2 10.172.169.192 datanode, NodeManager
Hadoop3 10.172.169.193 datanode, NodeManager
Hadoop4 10.172.169.194 datanode, NodeManager
2. Each machine is installed with javajdk and corresponding environment variables are configured. The installation path and java environment variables of each machine must be consistent.
3. Disable the Firewall
Switch to the root account
Enable: chkconfigiptables on
Close: chkconfigiptables off
It takes effect permanently after restart
4. Configure the/etc/host file for each server.
Under the root account, open the/etc/host file and add IP Address Resolution
Hadoop1 10.172.169.191
Hadoop2 10.172.169.192
Hadoop3 10.172.169.193
Hadoop4 10.172.169.194
5. Configure ssh login without a password
(1). On the namenode Machine
Cd/home/myhadoop
Ssh-keygen-t rsa
Press ENTER
(2) import the public key to the local authentication File
Cat ~ /. Ssh/id_rsa.pub> ~ /. Ssh/authorized_keys
(3) import the public key to another datanode node authentication File
Scp ~ /. Ssh/authorized_keys myhadoop@10.172.169.192:/home/myhadoop/. ssh/authorized_keys
Scp ~ /. Ssh/authorized_keys myhadoop@10.172.169.193:/home/myhadoop/. ssh/authorized_keys
Scp ~ /. Ssh/authorized_keys myhadoop@10.172.169.194:/home/myhadoop/. ssh/authorized_keys
(4). Modify the file permissions on all machines
Chmod 700 ~ /. Ssh
Chmod 600 ~ /. Ssh/authorized_keys
(5). Test whether you can log on without a password through ssh.
If namenode can log on to each datanode machine without a password, the configuration is successful.
Build a Hadoop environment on Ubuntu 13.04
Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1
Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)
Configuration of Hadoop environment in Ubuntu
Detailed tutorial on creating a Hadoop environment for standalone Edition
Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)