Hadoop fully distributed environment construction

I. Preparations
Lab environment: Vmware virtualizes three hosts, with the system CentOS_6.4_i386
Used Software: Hadoop-1.2.1-1.i386.rpm, jdk-7u9-linux-i586.rpm
Host planning:
IP address host name role
192.168.2.22 master. flyence. tk NameNode, JobTracker
192.168.2.42 datanode. flyence. tk DataNode, TaskTracker
192.168.2.32 snn. flyence. tk SecondaryNameNode

1. Run the hostname command to modify the Host Name and modify the host in/etc/sysconfig/network.
Take the master node as an example.
[Root @ localhost ~] # Hostname master. flyence. tk
[Root @ localhost ~] # Vim/etc/sysconfig/network
[Root @ localhost ~] # Logout
-
The content in/etc/sysconfig/network is as follows:
NETWORKING = yes
HOSTNAME = master. flyence. tk

2. Record the IP address and Host Name of the three hosts in/etc/hosts.

3. Add a hadoop user to the three hosts and set the password.

# Useradd hadoop
# Echo "hadoop" | passwd -- stdin hadoop

4. hadoop users on the master node can log on to other nodes through key-based authentication, so as to start the process and perform additional management work such as monitoring.
[Root @ master ~] # Su-hadoop
[Hadoop @ master ~] $ Ssh-keygen-t rsa-p''
[Hadoop @ master ~] $ Ssh-copy-id-I. ssh/id_rsa.pub hadoop@datanode.flyence.tk
[Hadoop @ master ~] $ Ssh-copy-id-I. ssh/id_rsa.pub hadoop@snn.flyence.tk

Ii. Install JDK

Install the three hosts. Repeat the following steps three times:

[Root @ master ~] # Rpm-ivh jdk-7u9-linux-i586.rpm

Edit/etc/profile. d/java. sh and add the following content to the file:
Export PATH =/usr/java/latest/bin: $ PATH

Switch to the hadoop user and run the following command to test whether the jdk environment is ready.
[Hadoop @ master ~] $ Java-version
Java version "1.7.0 _ 09"
Java (TM) SE Runtime Environment (build 1.7.0 _ 09-b05)
Java HotSpot (TM) Client VM (build 23.5-b02, mixed mode, sharing)

3. Install Hadoop

Hadoop must be installed on each node in the cluster.
[Root @ master ~] # Rpm-ivh hadoop-1.2.1-1.i386.rpm

Switch to hadoop user to verify if Hadoop installation is complete
[Hadoop @ master ~] $ Hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2-r 1503152
Compiled by mattf on Mon Jul 22 15:17:22 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using/usr/share/hadoop/hadoop-core-1.2.1.jar.

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)

1 2 3 Next Page