Build the Hadoop2.2.0 environment in the OpenStack Environment

OpenStack has become the first choice for many cloud computing vendors to build a private cloud. Many academic institutions also use OpenStack to build a small-scale test environment for students' experiments. Here we will share the process of using OpenStack Virtual Machine to build the Hadoop2.2.0 environment.

1. Prepare the VM Environment

OpenStack version: Folsom

A. Launch three test VMS with Ubuntu-12.04.2-x86_64 Operating System

B. Configure the IP address. In OpenStack of version F, the network uses the FlatDHCP mode to allow the virtual machine to obtain the Fixed IP address in the 10.0.x.x segment. Therefore, you need to configure the/etc/hosts file in the virtual machine.

# Vim/etc/hosts

127.0.0.1 localhost. localdomain
10.0.0.225 hdp-server-01
10.0.1.19 hdp-server-02
10.0.1.17 hdp-server-03

C. Use root to create a user yarn on each machine and use the same password.

# Useradd-m-s/bin/bash yarn
# Passwd yarn
Enter new UNIX password:
Retype new UNIX password:
Passwd: password updated successfully

D. Set ssh password-less mutual access

# Each machine
$ Su yarn
$ Cd ~
$ Ssh-keygen-t rsa
$ Cat. ssh/id_rsa.pub>. ssh/authorized_keys
# You can use ssh localhost to test whether password-free access is allowed.
# You can copy the. ssh/authorized_keys content to the. ssh/authorized_keys file of the other party.

E. Use the yarn account to access each other through the host name entered in the/etc/hosts file, and verify that there is no password for logon.

Because a 64-bit operating system is used, files downloaded from the official website cannot be directly installed and must be compiled manually. The compilation process is as follows:

2. Compile Hadoop2.2.0

A. Configure the JDK environment variable. Assume that the jdk folder is/usr/java/jdk1.7.0 _ 45.

Su yarn
# Vim ~ /. Bashrc
# Append write

Export JAVA_HOME =/usr/local/java/jdk1.7.0 _ 45
Export JRE_HOME =$ {JAVA_HOME}/jre
Export CLASSPATH =. :$ {JAVA_HOME}/lib :$ {JRE_HOME}/lib
Export PATH =$ {JAVA_HOME}/bin: $ PATH

$ Source ~ /. Bashrc # Make the settings take effect

 

 

Install and deploy Openstack on Ubuntu 12.10

Ubuntu 12.04 OpenStack Swift single-node deployment Manual

OpenStack cloud computing quick start tutorial

Deploying OpenStack for enterprises: what should be done and what should not be done

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

1 2 3 4 5 Next Page