Learning Prelude to Hadoop (ii) configuration of the--hadoop cluster

Preface:

The configuration of a Hadoop cluster is a fully distributed Hadoop configuration.

the author's environment:

Linux:centos 6.6 (Final) x64
Jdk:java Version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-bit Server VM (build 24.75-b04, Mixed mode)
SSH:OPENSSH_5.3P1, OpenSSL 1.0.1e-fips 2013
hadoop:hadoop-1.2.1

steps:

Note: the experiment in this paper is based on the pseudo-distributed Hadoop environment. by default, your system can already install pseudo-distributed Hadoop. If your system is not yet installed with pseudo-distributed Hadoop, refer to the learning prelude to Hadoop-Installing and configuring Hadoop on Linux-to complete the configuration.

1. First create the same user on three hosts

2. Set/etc/hosts and/etc/hostname on each of the three hosts

$ vim/etc/hosts

$ vim /etc/hostname

3. Install OpenSSH on these three hosts and configure SSH to login with no password.

I have explained in detail in my previous blog about the part of password-free login on a single machine. If you have a problem with your configuration that you can't password-free, there are specific practices in the previous blog that you don't have to repeat.

Copy the Authorized_keys from master to two slave as follows:

$ SCP Authorized_keys slave1:~/.ssh/

$ SCP Authorized_keys slave2:~/.ssh/

4. Configure the Hadoop files for the three hosts.

$ vim Core-site.xml

$ vim Hdfs-site.xml

$ vim Mapred-site.xml

$ vim Masters

$ VIM Slaves

Now that the configuration of Hadoop is basically complete, you can

5. Start Hadoop

Bin/hadoop Namenode-format

bin/start-all.sh

6. Verification

You can view the cluster status via the following command or by accessing http://master:50030 and http://master:50070

Hadoop Dfsadmin-report

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced. http://blog.csdn.net/lemon_tree12138

Learning Prelude to Hadoop (ii) configuration of the--hadoop cluster