Hadoop pseudo-distributed installation on a Linux host

1. Install JDK

sudo apt-get install openjdk-6-jdk

2. Configure Java environment variables

Enter the following command:

sudo gedit /etc/profile

Open the profile and add the following content at the bottom of the file:

# set java homeexport JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-i386/export CLASSPATH=".:JAVA_HOME/lib:$CLASSPATH"export PATH="$JAVA_HOME:$PATH"

Note that you can add the SDK according to the Java path you have installed.

Ls/usr/lib/JVM/to view the installed JDK path.

Verify that JDK is successfully installed:

java  -version

3. Configure SSH password-free Login

A. Install SSH and enter the following command:

sudo apt-get  install ssh

B. you can log on to the local machine without a password.

Check whether there is a folder. Ssh.

Command:

Ls-L/home/U (your user directory)

C. Create the id_dsa id_dsa.pub file, which is an SSH private key and a public key.

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

D. append id_dsa.pub to the authorization key.

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

D. verify whether the request is successful.

 

ssh localhost

Display
Welcome to Ubuntu 12.10 (GNU/Linux 3.2.0-29-generic-pae i686)

* Documentation: https://help.ubuntu.com/

Last login: Sun Apr 21 11:16:27 2013 from daniel-optiplex-320.local

4. hadoop Installation

A. Download hadoop

Click Open Link

B. Decompress hadoop

tar xzvf hadoop-1.0.4-bin.tar.gz  -C /home/user

Decompress the package to your user directory.

C. Pseudo-distributed hadoop Configuration

Go to the hadoop directory under/home/user.

CD Conf, go to the configuration folder, and modify it.

Modify the JDK file hadoop-env.sh configuration.

export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-i386/

Modify file core-site.xml

<configuration> <property><name>fs.default.name</name><value>hdfs://localhost:9000</value> </property></configuration>

Modify file hdfs-site.xml

<configuration> <property><name>dfs.replication</name><value>1</value> </property></configuration>

Modify file mapred-site.xml

<configuration>  <property><name>mapred.job.tracker</name><value>localhost:9001</value>  </property></configuration>

5. Start hadoop

First, add the hadoop bin directory to the path system environment variable.
 

sudo  vim /etc/profile

Add the following in the last line:

export PATH=/home/user/hadoop-1.0.4/bin/:$PATH

Note that this path is the bin directory path under the hadoop directory after pressurization.
After the modification, enter the following command:

source  /etc/profile

A. format the file system.

Hadoop namenode-format B, start hadoop boot close all services start-all.sh/stop-all.sh start close HDFS: start-dfs.sh/stop-dfs.sh start close mapreduce: start-mapred.sh/stop-mapred.sh C, view processes with JPs command, ensure that namenode, datanode, jobtracker, and tasktracker are available.
You can also use a browser to verify that hadoop is successfully installed.
Mapreduce web page
Http: // localhost: 50030
HDFS web page
Http: // localhost: 50070

If a java. Io. ioexception: Too program open files error occurs when the correct method is to add this line to/etc/security/limits. conf:
-Nofile 8192