Install Hadoop2.4.0 in Ubuntu 14.04 (standalone Mode)

Install Hadoop2.4.0 in Ubuntu 14.04 (standalone Mode)

1. Create a Hadoop group and hadoop user in Ubuntu

Add a hadoop user group and add a hadoop user in this group. We will use this user in subsequent hadoop operations.

1. Create a hadoop User Group

2. Create a hadoop user

Sudo adduser-ingroup hadoop

After you press enter, you will be prompted to enter a new UNIX Password, which is the new user's hadoop password. Enter.

If you do not enter the password, enter the password again, that is, the password cannot be blank.

Check whether the information is correct. If no problem occurs, enter Y and press Enter.

3. add permissions for hadoop users

Input: sudo gedit/etc/sudoers

Press enter to open the sudoers file.

Grant hadoop users the same permissions as root users

2. log on to the Ubuntu system with the newly added hadoop user

3. Install ssh

Sudo apt-get install openssh-server

After the installation is complete, start the service

Sudo/etc/init. d/ssh start

Check whether the service is properly started: ps-e | grep ssh

Set password-free logon to generate the private key and Public Key

Ssh-keygen-t rsa-P ""

In this case, two files are generated under/home/hadoop/. ssh: id_rsa and id_rsa.pub. The former is the private key and the latter is the public key.

Next we append the public key to authorized_keys, which saves all the public key content that allows users to log on to the ssh client as the current user.

Cat ~ /. Ssh/id_rsa.pub> ~ /. Ssh/authorized_keys

Log on to ssh

Ssh localhost

Exit

Exit

4. Install the Java environment

Sudo apt-get install openjdk-7-jdk

View the installation result and enter the command: java-version. The result is as follows: the installation is successful.

5. Install hadoop2.4.0

1. Download http://mirror.bit.edu.cn/apache/hadoop/common/

2. Installation

Extract

Sudo tar xzf hadoop-2.4.0.tar.gz

If we want to install hadoop under/usr/local

Copy to/usr/local/, and the folder is hadoop

Sudo mv hadoop-2.4.0/usr/local/hadoop

Grant the user the read and write permissions on the folder.

Sudo chmod 774/usr/local/hadoop

3. Configuration

1) configuration ~ /. Bashrc

Before configuring this file, you need to know the Java installation path to set the JAVA_HOME environment variable. You can use the following command line to view the installation path

Update-alternatives--config java

The execution result is as follows:

The complete path is

/Usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java

We only take the previous part/usr/lib/jvm/java-7-openjdk-amd64

Configure the. bashrc File

Sudo gedit ~ /. Bashrc

This command will open the editing window of the file, append the following content at the end of the file, save it, and close the editing window.

# HADOOP VARIABLES START

Export JAVA_HOME =/usr/lib/jvm/java-7-openjdk-amd64

Export HADOOP_INSTALL =/usr/local/hadoop

Export PATH = $ PATH: $ HADOOP_INSTALL/bin

Export PATH = $ PATH: $ HADOOP_INSTALL/sbin

Export HADOOP_MAPRED_HOME = $ HADOOP_INSTALL

Export HADOOP_COMMON_HOME = $ HADOOP_INSTALL

Export HADOOP_HDFS_HOME = $ HADOOP_INSTALL

Export YARN_HOME = $ HADOOP_INSTALL

Export HADOOP_COMMON_LIB_NATIVE_DIR = $ HADOOP_INSTALL/lib/native

Export HADOOP_OPTS = "-Djava. library. path = $ HADOOP_INSTALL/lib"

# HADOOP VARIABLES END

The final result is as follows:

Run the following command to make the added environment variables take effect:

Source ~ /. Bashrc

2) edit/usr/local/hadoop/etc/hadoop/hadoop-env.sh

Run the following command to open the file editing window.

Sudo gedit/usr/local/hadoop/etc/hadoop/hadoop-env.sh.

Find the JAVA_HOME variable and modify it as follows:

Export JAVA_HOME =/usr/lib/jvm/java-7-openjdk-amd64

The modified hadoop-env.sh file is as follows:

6. WordCount Test

After the installation is completed in standalone mode, run the WordCount built-in hadoop instance to verify whether the installation is successful.

/Usr/local/hadoop path to create the input Folder

Mkdir input

Beibeireadme.txt to input

Cp README.txt input

Execute WordCount

Bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.4.0-sources.jar org. apache. hadoop. examples. WordCount input output

Execution result:

Run cat output/* to view the character statistics.

Install and configure Hadoop2.2.0 on CentOS

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)