# Hadoop # single-host (pseudo distribution) installation and testing

TIPS: The deployment is completely performed by the root user. The system is ubuntu14.04, and the original openjdk is replaced by Oracle jdk7_u55.

Installation Process:

Install Java:

Download the package from your host's FTP Server:

cd /usr/lib/jvmwget ftp://192.168.42.110/jdk-7u55-linux-x64.tar.gz tar xvf jdk-7u55-linux-x64.tar.gzmv jdk1.7.0_55/ java-7-sun

Edit bashrc:

VI ~ /. Bashrc # Add the following content: Export java_home =/usr/lib/JVM/java-7-sunexport jre_home =$ {java_home}/jreexport classpath =.: $ {java_home}/lib :$ {jre_home}/libexport path =$ {java_home}/bin: $ path # source ~ /. Bashrc

Because there is also a default openjdk, you need to set the default JDK to the just-installed Oracle JDK:

update-alternatives --install /usr/bin/java java /usr/lib/jvm/java-7-sun/bin/java 300update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/java-7-sun/bin/javac 300update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/java-7-sun/bin/jar 300 update-alternatives --install /usr/bin/javah javah /usr/lib/jvm/java-7-sun/bin/javah 300 update-alternatives --install /usr/bin/javap javap /usr/lib/jvm/java-7-sun/bin/javap 300

Run:

update-alternatives --config java

The following prompt is displayed:

[email protected]:/usr/lib/jvm# update-alternatives --config java There are 2 choices for the alternative java (providing /usr/bin/java).  Selection    Path                                            Priority   Status------------------------------------------------------------  0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      auto mode  1            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      manual mode* 2            /usr/lib/jvm/java-755-sun/bin/java               300       manual modePress enter to keep the current choice[*], or type selection number:

Select a number based on your needs.

Test successful:

[email protected]:/usr/lib/jvm# java -versionjava version "1.7.0_55"Java(TM) SE Runtime Environment (build 1.7.0_55-b13)Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)

Install SSH:

apt-get install ssh

Try SSH local. If you need a password, you need to set a local login without a password:

ssh-keygen -t dsa -P ‘‘ -f ~/.ssh/id_dsacat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

Install rsync:

apt-get install rsync

Configure hadoop:

tar -xvf  hadoop-1.0.3.tar.gzcd hadoop-1.0.3

Edit the conf/hadoop-env.sh file to modify the content of this export:

# Export java_home =/usr/lib/j2sdk1.5-Sun (original content) Export java_home =/usr/lib/JVM/Java-7-sun

Modify CONF/core-site.xml:

<configuration><property><name>fs.default.name</name><value>hdfs://localhost:9000</value></property></configuration>

Modify CONF/hdfs-site.xml:

<configuration><property><name>dfs.replication</name><value>1</value></property><property><name>hadoop.tmp.dir</name><value>/home/work/hadoop_tmp</value></property></configuration>

Modify CONF/mapred-site.xml:

<configuration><property><name>mapred.job.tracker</name><value>localhost:9001</value></property></configuration>

Format namenode:

bin/hadoop namenode –format

Start and check the running status:

# Run bin/start-all.sh # Check status JPs # output 5146 jps4538 tasktracker4312 jobtracker4015 datanode4228 secondarynamenode3789 namenode

If the status is correctly displayed, the operation is correct.

Test use:

[1] Web interface:

Http: // localhost: 50030

You can view the running status of jobtracker.

Http: // localhost: 50060

You can view the running status of tasktracker.

Http: // localhost: 50070

You can view the status of namenode and the entire Distributed File System, and view files and logs in the distributed file system.

[2] test with the wordcount program included in the hadoop1.0.3 directory:

Create two input files file01 and file02 on the local disk:

echo "Hello World Bye World" > file01  echo "Hello Hadoop Goodbye Hadoop" > file02

Create an input directory in HDFS:

hadoop fs –mkdir input

Copy file01 and file02 to HDFS:

hadoop fs –copyFromLocal file0* input

Execute wordcount:

hadoop jar hadoop-examples-1.0.3.jar wordcount input output

Then, view the result:

Bin/hadoop FS-ls # view the list of existing files

Hadoop FS-cat output/part-r-00000 # view results

The result is: all words are counted.

Bye 1  Goodbye 1 Hadoop 2 Hello 2 World 2

References:

Hadoop pseudo-distribution installation process: hadoop standalone environment Setup Guide (UBUNTU)

Http://www.aboutyun.com/thread-6487-1-1.html

After the cluster is installed, how to test and use the cluster-hadoop single-host (pseudo distribution)

Http://www.aboutyun.com/thread-6777-1-1.html

Cloud technology basics: The role of clusters to build SSH and the meaning of these commands

Http://www.aboutyun.com/thread-6977-1-1.html

How to install and configure JDK 7 in Ubuntu 11.04

Http://www.aboutyun.com/thread-6750-1-1.html

# Hadoop # single-host (pseudo distribution) installation and testing