Ubuntu installs Hadoop (standalone mode)

First, create Hadoop groups and Hadoop users under ubuntu

Increase the Hadoop user group while adding Hadoop users to the group, and we'll use that user when it comes to Hadoop operations .

1. Create a Hadoop user group

2, create a Hadoop user

sudo adduser-ingroup Hadoop Hadoop

You will be prompted to enter a new UNIX password, which is the password for the new user Hadoop , enter the return.

If you do not enter a password, you will be prompted to enter the password again, that is, the password cannot be empty.

Finally confirm the information is correct, if no problem, enter Y, carriage return.

3, Add permissions for Hadoop users

Input: sudo gedit/etc/sudoers

Enter, open the Sudoers file

Give the Hadoop user the same permissions as the root user

Ii. sign in to the Ubuntu system with the newly added Hadoop user

Third, install SSH

sudo apt-get install Openssh-server

After the installation is complete, start the service

Sudo/etc/init.d/ssh start

To see if the service started correctly: PS-E | grep ssh

Setting up password-free logins, generating private and public keys

Ssh-keygen-t rsa-p ""

Two files are generated under/home/hadoop/.ssh: Id_rsa and Id_rsa.pub, which is the private key and the latter is the public key.

Next we append the public key to Authorized_keys, which saves all the public key content that allows the user to log on to the SSH client as the current user.

Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Login to SSH

SSH localhost

Exit

Exit

Iv. installation of the Java environment

sudo apt-get install OPENJDK-7-JDK

To view the installation results, enter the command: Java-version, the results are as follows to indicate that the installation was successful.

v. Installation of hadoop2.4.01, official website download http://mirror.bit.edu.cn/apache/hadoop/common/

2, installation

Extract

sudo tar xzf hadoop-2.4.0.tar.gz

If we're going to install Hadoop under the/usr/local,

Copy to/usr/local/, folder for Hadoop

sudo mv Hadoop-2.4.0/usr/local/hadoop

Give users read and write access to this folder

sudo chmod 774/usr/local/hadoop

3. Configuration

1) Configuration ~/.BASHRC

Before you configure this file, you need to know the Java installation path to set the JAVA_HOME environment variable, you can use the following command line to view the installation path

Update-alternatives--config Java

The results of the implementation are as follows:

The full path is

/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java

We only take the previous part/USR/LIB/JVM/JAVA-7-OPENJDK-AMD64

Configure the. bashrc file

sudo gedit ~/.BASHRC

The command opens the edit window for the file, appends the following to the end of the file, and then saves and closes the editing window.

#HADOOP VARIABLES START

Export JAVA_HOME=/USR/LIB/JVM/JAVA-7-OPENJDK-AMD64

Export Hadoop_install=/usr/local/hadoop

Export path= $PATH: $HADOOP _install/bin

Export path= $PATH: $HADOOP _install/sbin

Export Hadoop_mapred_home= $HADOOP _install

Export Hadoop_common_home= $HADOOP _install

Export Hadoop_hdfs_home= $HADOOP _install

Export Yarn_home= $HADOOP _install

Export hadoop_common_lib_native_dir= $HADOOP _install/lib/native

Export hadoop_opts= "-djava.library.path= $HADOOP _install/lib"

#HADOOP VARIABLES END

The final results are as follows:

Execute the following to make the added environment variable effective:

SOURCE ~/.BASHRC

2) Edit/usr/local/hadoop/etc/hadoop/hadoop-env.sh

Execute the following command to open the edit window for the file

sudo gedit/usr/local/hadoop/etc/hadoop/hadoop-env.sh

Locate the Java_home variable and modify the variable as follows

Export JAVA_HOME=/USR/LIB/JVM/JAVA-7-OPENJDK-AMD64

The modified hadoop-env.sh file resembles the following :

VI. WordCount Test

Stand-alone mode installation is complete, following the implementation of Hadoop's own instance wordcount verifying that the installation was successful

Create input folder under/usr/local/hadoop path

mkdir input

Copy README.txt to input

CP README.txt Input

Executive WordCount

Bin/hadoop Jar Share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.4.0-sources.jar Org.apache.hadoop.examples.WordCount Input Output

Execution Result:

Perform cat output/* to view character statistics results

Transferred from: http://www.cnblogs.com/kinglau/p/3794433.html

Ubuntu installs Hadoop (standalone mode)