Create a single-node environment for Hadoop 1.2.1 in Centos 6.5

Create a single-node environment for Hadoop 1.2.1 in Centos 6.5
Install and configure Java

• JDK Installation

# Chmod 777 jre-6u31-linux-x64-rpm.bin #./jre-6u31-linux-x64-rpm.bin

• Add java environment variables

# Vim/etc/profile # Add # set java environmentexport JAVA_HOME =/usr/java/jre1.6.0 _ 31 export CLASSPATH =.: $ JAVA_HOME/jre/lib/rt. jar: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jarexport PATH = $ PATH: $ JAVA_HOME/bin

Valid java variables:

# Source/etc/profile # java-version

• Add a hadoop user

Create the same directory on all machines. You can also create the same user. It is best to use the home path of the user as the hadoop installation path. Installation paths are:/home/hadoop/hadoop-1.2.1

# Useradd hadoop # passwd hadoop # The password is also hadoop

• Login without a password

# su - hadoop$ ssh-keygen -t rsa$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys$ chmod 0600 ~/.ssh/authorized_keys

• Download and install hadoop

# Su-hadoop # wget http://apache.stu.edu.tw/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz # tar-zxvf hadoop-1.2.1.tar.gz

My installation directory is:

/Home/hadoop/hadoop-1.2.1

For ease, use commands such as the hadoop command or start-all.sh to add the following to/etc/profile on the modify node:

Export HADOOP_HOME =/home/hadoop/hadoop-1.2.1

Export PATH = $ PATH: $ HADOOP_HOME/bin

After modification, execute source/etc/profile to make it take effect.

• Configure Hadoop

Configure the conf/hadoop-env.sh file and add:

Export JAVA_HOME =/usr/java/jre1.6.0 _ 31

Here, we will change it to your jdk installation location.

• Test hadoop Installation

Run the built-in hadoop example to check whether hadoop is successfully installed cd $ HADOOP_HOME/bin.

/Home/hadoop/hadoop-1.2.1/bin/hadoop jar./hadoop-1.2.1/hadoop-examples-1.2.1.jar wordcount./hadoop-1.2.1/conf/tmp/out

• Modify configuration file

Vi/home/hadoop/hadoop-1.2.1/conf/core-site.xml <configuration> <property> <name> fs. default. name </name> <value> hdfs: // localhost: 9000 </value> </property>

</Configuration>

Vi/home/hadoop/hadoop-1.2.1/conf/hdfs-site.xml <configuration> <property> <name> dfs. replication </name> <value> 1 </value> </property> </configuration>Vi/home/hadoop/hadoop-1.2.1/conf/mapred-site.xml <configuration> <property> <name> mapred. job. tracker </name> <value> localhost: 8021 </value> </property> </configuration>

• Format HDFS

Hadoop namenode-format

• Start the service

Start-dfs.shstart-mapred.sh

• Stop Service

Stop-dfs.shstop-mapred.sh

• Verify

The default port for Hadoop NameNode is 50070.

Access port 50090 to obtain information about secondary namenode.

Access port 50075 to obtain information about DataNode.