Install Hadoop2.2.0

Part 1 download Hadoop2.2

For Hadoop, download the latest version Hadoop2.2 from the Apache official website. Currently, linux32-bit system executable files are provided officially. (The FAQ at the bottom solves the problem of 64-bit)

: Http://apache.claz.org/hadoop/common/hadoop-2.2.0/

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

 

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

 

Configuration of Hadoop environment in Ubuntu

 

Detailed tutorial on creating a Hadoop environment for standalone Edition

 

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

 

Part 2 Cluster Environment Construction

1. Here we build a cluster composed of three machines:

IP user/passwd hostname

192.168.101.114 root/test123 hadoop1 nn/snn/rm GNU/Linux x86_64

192.168.101.115 root/test123 hadoop2 dn/nm GNU/Linux x86_64

192.168.101.116 root/test123 hadoop3 dn/nm GNU/Linux x86_64

1.1 The above columns are IP, user/passwd, hostname, and roles (namenode, secondary namenode, datanode, resourcemanager, nodemanager) in the cluster)

2. Modify the/etc/hosts file and add the ip ing between the ip addresses and hostnames of the three hosts.

192.168.101.114 hadoop1

192.168.101.115 hadoop2

192.168.101.116 hadoop3

3. SSH password-less login from hadoop1 to hadoop2 and hadoop3

3.1 Install ssh

Generally, ssh commands are installed by default. If not, or the version is old, you can reinstall it:

Sodu apt-get install ssh

3.2 set local Login Without Password

After the installation is complete ~ Directory (the current user's main directory, that is,/home/hduser) generates a hidden folder. ssh (ls-a can view hidden files ). If you do not have this file, create it yourself (mkdir. ssh ).

The procedure is as follows:

Run the following command under the hadoop Installer:

1. Enter the. ssh folder

2. ssh-keygen-t rsa followed by a carriage return (generate a key)

3. append id_rsa.pub to the authorization key.

> Cat id_rsa.pub> authorized_keys

Switch to the root user:

4. Restart the SSH service command to make it take effect: service sshdrestart

Now you can log on to the ssh localhost without a password.

[Note]: The preceding operations must be performed on each machine.

3.3 set Remote Login Without Password

After each machine is executed, copy id_rsa.pub as the namenode machine to the other two machines and change it to id_rsa_1.pub.

Cat id_rsa.pub_1> authorized_keys

Use hadoop1 for verification after restart:

> Ssh hadoop2

> Exit

> Ssh hadoop3

> Exit

In this way, the channel is opened.

4. Install jdk (the JAVA_HOME path of each machine is recommended to be the same)

Note: Download jdk and install it on your own, instead of directly installing it through the source (apt-get install)

4.1 download jkd (http://www.Oracle.com/technetwork/java/javase/downloads/index.html)

4.1.1 for 32-bit systems, you can download the following two Linuxx86 versions (uname-a View System Version)

4.1.2 64-bit system download Linux x64((x64.rpmand x64.tar.gz)

4.2、install jdk(.tar.gz, 32-bit system as an example)

Installation Method reference http://docs.oracle.com/javase/7/docs/webnotes/install/linux/linux-jdk.html

4.2.1 select the location where you want to install java, such as in the/usr/directory, and create a folder named java (mkdirjava)

4.2.2 move the jdk-7u40-linux-i586.tar.gz file to/usr/java

4.2.3 unzip: tar -zxvfjdk-7u40-linux-i586.tar.gz

4.2.4 Delete jdk-7u40-linux-i586.tar.gz (to save space)

Now, after jkd is installed, configure the environment variables below

4.3 open/etc/profile (vim/etc/profile)

Add the following content at the end:

JAVA_HOME =/usr/java/jdk1.7.0 _ 40 (the version number 1.7.40 must be modified based on the download details)

CLASSPATH =.: $ JAVA_HOME/lib. tools. jar

PATH = $ JAVA_HOME/bin: $ PATH

Export JAVA_HOMECLASSPATH PATH

4.4. source/etc/profile

4.5 verify whether the installation is successful: java-version

[Note] Each machine performs the same operation and finally installs java in the same path (not required, but this will make subsequent configuration much more convenient)

5. Disable the firewall for each machine

RedHat:

/Etc/init. d/iptables stop disable the firewall.

Chkconfig iptables off disable startup.

 

 

For more details, please continue to read the highlights on the next page:

1 2 Next Page