Hadoop cluster Environment Setup

1 Hadoop Cluster planning

1.1 Total of A, B, C three machines;

1.2 A as master,b as slave1,c as Slave2;

1.3 IP &http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; a:192.168.1.103; b:192.168.1.104; c:192.168.1.101;

1.4 os:centos6.2 (64bit)

2 Hadoop Cluster setup steps

2.1 Create a Hadoop user and give the user root privileges (done on the master machine)

#创建hadoop用户

Useradd Hadoop

#给用户hadoop添加密码

passwd Hadoop

#付给hadoop用户root权限

VI/ETC/PASSWD the user ID, group ID of the Hadoop user to 0, such as Hadoop:x:0:0::/home/hadoop:/bin/bash (remember to switch to root first, other users estimate that the file does not have permissions)

2.2 Switch to Hadoop user, enter Hadoop user home directory (on Master machine)

#切换到hadoop用户

Su Hadoop

Enter a password as prompted (if you want to switch from a regular user to Hadoop, you need a password; If you switch from root to Hadoop, you don't have to enter a password, there's no such a step)

#进入hadoop的主目录

Cd/home/hadoop

2.3 Installing the JDK environment (Hadoop is a Java-written tool that requires JDK) (on Master machine)

Yum Install openjdk* (yum is a tool for installing software on CentOS to find software, remove software, update software, etc.)

2.4 Install OpenSSH (it is required for communication between Hadoop nodes and is free of charge relative to SSH) (done on Master machine)

#安装openssh

Yum Install openssh*

#生存ssh密钥

Ssh-keygen-t Rsa-p '

#进入. SSH folder

Cd/root/.ssh

#将公钥加到认证的公钥文件中

Cat Id_rsa.pub>>authorized_keys

2.5 Download the Hadoop installation package and configure it accordingly (on Master machine)

#下载hadoop -1.0.4.tar.gz

#解压hadoop -1.0.4.tar.gz to/home/hadoop.

The Conf folder in the #hadoop -1.0.4 folder is the file we need to configure (see address for details)

http://download.csdn.net/detail/lovemelovemycode/5634581

2.6 Configuring hosts and hostname (on master)

#下载这个文件

http://download.csdn.net/detail/lovemelovemycode/5634599

2.7 Repeat steps 2.1, 2.2, 2.3 on slave1 and Slave2

2.8 Copy the corresponding files on master to Slave1\slave2 (on master)

Scp-r. SSH hadoop@slave1:/root

Scp-r. SSH hadoop@slave2:/root

Scp-r/home/hadoop/hadoop-1.0.4 Hadoop@slave1:/home/hadoop

Scp-r/home/hadoop/hadoop-1.0.4 Hadoop@slave2:/home/hadoop

Scp-r hosts Hadoop@slave1:/etc

Scp-r/etc/hostname hadoop@slave2:/etc

2.9 Closing the master, Slave1, and slave2 firewalls

#暂时关闭防火墙

/etc/init.d/iptables stop

#永久关闭防火墙

Chkconfig–level iptables off

2.10 Format Namenode\datanode (on master)

Bin/hadoop Namenode-format

Bin/hadoop Datanode-format

2.11 Start Hadoop (on master)

# into hadoop-1.0.4

bin/start-all.sh

3 Hadoop cluster to build a successful verification method

3.1 Validation via Java JPS command

http://download.csdn.net/detail/lovemelovemycode/5634649

3.2 Verification through http://master:50070 and http://master:50030