Install Hadoop2.2 In pseudo distribution mode in CentOS 7

Install Hadoop2.2 In pseudo distribution mode in CentOS 7

Here we will not introduce the installation of CentOS. For details, refer to the following connection and directly go to Hadoop2.2 pseudo distribution mode for installation.

Install CentOS7.0 on a vmwarevm

64-bit CentOS 6.5 Installation Method

64-bit CentOS 6.4 dual-system installation in Win7

Install Windows 8 and CentOS on UEFI + GPT

Windows 8.1 and CentOS6.5 dual-system installation

1. Download the installation package
1. Download JDK1.7
The current JDK version is jdk1.8.0 _ 25. Here, JDK 1.7.0 _ 67 is downloaded.

Download Page: http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html

1. 2. Download Hadoop2.2
The latest version of Hadoop is hadoop2.5. Hadoop2.2 is downloaded here.

Download Page: http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-2.2.0/

2. Install Hadoop2.2
2. 1. Configure the installation environment
2.1.1. Create a hadoop user
Log on to CentOS with the root account and perform the following operations:

#/Usr/sbin/groupaddhadoop

Create a hadoop user, affiliated to the hadoop Group

#/Usr/sbin/useradd-ghadoophadoop

Modify the password of A hadoop user to hadoop:

# Passwd hadoop

2.1.2. Configure hadoop users
Edit the/etc/sysconfig/network-scripts/ifcfg-eth0 file to set static IP address information.

# Vi/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE = eth0

ONBOOT = yes

BOOTPROTO = static

IPADDR = 192.168.15.136

NETMASK = 255.255.255.0

GATEWAY = 192.168.15.2

Make the modified parameters take effect immediately:

Close first:

#/Sbin/ifdown eth0

Restart:

#/Sbin/ifup eth0

Edit the/etc/hostname file and set the Host Name

# Vi/etc/hostname

Master. Hadoop

Edit the/etc/hosts file and set the ing between host names and IP addresses.

# Vi/etc/hosts

192.168.15.136Master.Hadoop

2.1.3. Create a Software Directory
Create the u01 directory for installing hadoop software.

Mkdir-p/u01/hadoop

Chown-Rhadoop: hadoop/u01/hadoop/

Chmod-R775/u01/hadoop/

Hadoop data storage directory

Mkdir-p/data/hadoop

Chown-R hadoop: hadoop/data/hadoop

Chmod-R 775/data/hadoop

All of the above operations are performed with the root account. Next, log on to the hadoop user and start to install hadoop2.2.

2.1.4. Upload the installation package
Log on to CentOS as a hadoop user and use the FTP tool to upload the installation package to the/u01/hadoop/directory.

2. Install JDK
Since JDK is an installation-free version, you only need to extract it to the specified directory to complete the installation, and install JDK under the/u01/hadoop/jdk directory.

# Tar-zxvf jdk-7u67-linux-x64.tar.gz

# Mv jdk1.7.0 _ 67 // u01/hadoop/jdk

Configure JDK Environment Variables

Vi. bash_bash_profile

Export JAVA_HOME =/u01/hadoop/jdk
Export CLASSPATH =.: $ JAVA_HOME/lib/tools. jar: $ JAVA_HOME/lib/rt. jar
Export PATH = $ JAVA_HOME/bin: $ PATH

(Note: When configuring the PATH environment variable, You must place $ JAVA_HOME/bin in front of $ PATH. In this way, the default JDK is not used)

Run the source command to immediately activate the JDK environment variable.

# Source. bash_profile

Check whether JDK is successfully installed!

# Java-version
Java version "1.7.0 _ 67"
Java (TM) SE Runtime Environment (build 1.7.0 _ 67-b01)
Java HotSpot (TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

The above information indicates that JDK is successfully installed!

2. 3. Install Hadoop2.2
Hadoop has three operating modes: Standalone mode, pseudo-distributed mode, and fully distributed mode. Pseudo-distributed is a cluster with only one node. Here we will only introduce the installation of pseudo-distributed systems.

-------------------------------------- Split line --------------------------------------

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

-------------------------------------- Split line --------------------------------------

2.3.1. Configure SSH password-free Login
Hadoop needs to use SSH to start the daemon process of the Slave node, and SSH is required even if the pseudo-distributed architecture is installed. I have installed SSH when installing CentOS. You only need to configure password-free login. If SSH is not installed, install SSH before proceeding to the following steps.

Run the following command in the home directory of the hadoop User:

$ Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa

(Note: ssh-keygen indicates that the key is generated;-t specifies the key type;-P provides the secret language;-f generates the key file)

$ Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys

$ Chmod600 ~ /. Ssh/authorized_keys

Verify password-free Login

$ Sshlocalhost
Last login: Sat Oct 25 07:42:43 2014

If this information is displayed, the SSH configuration is successful.

2.3.2.hadoop2.2 installation and configuration
The hadoop installation process is very simple. You only need to extract the package to the specified directory. In fact, the hadoop installation steps are mainly in the configuration file. Decompress the hadoop installation package to/u01/hadoop.

Tar-zxvf hadoop-2.2.0.tar.gz

Music hadoop-2.2.0/u01/hadoop

Main configuration of these five files: hadoop-env.sh, core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml

Go to the hadoop configuration file directory

Cd/u01/hadoop/etc/hadoop/

Specify the JDK home directory:

# Vi hadoop-env.sh

ExportJAVA_HOME =/u01/hadoop/jdk

For more details, please continue to read the highlights on the next page:

1 2 Next Page