Hadoop installation and configuration-pseudo Distribution Mode

1. Install

Here the installation of hadoop-0.20.2 as an Example

Install Java first. Refer to this

Download hadoop

Extract

tar -xzf hadoop-0.20.2

 

2. Configuration

Modify Environment Variables

Vim ~ /. Bashrcexport hadoop_home =/home/RTE/hadoop-0.20.2 # The directory location export path for actual hadoop decompression here = $ path: $ hadoop_home/binsource ~ /. Bashrc

Configure hadoop-env.sh

vim conf/hadoop-env.shexport JAVA_HOME=/home/rte/Software/java/jdk1.6.0_27

Configure CONF/core-site.xml, CONF/hdfs-site.xml, CONF/mapred-site.xml File

File: core-site.xml

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration>    <property>        <name>fs.default.name</name>        <value>hdfs://localhost:9000</value>    </property></configuration>

File: mapred-site.xml

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration>        <property>                <name>mapred.job.tracker</name>                <value>localhost:9001</value>         </property></configuration>

File: hdfs-site.xml

<?xml version="1.0"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Put site-specific property overrides in this file. --><configuration>        <property>                <name>dfs.replication</name>                <value>2</value>        </property></configuration>

 

3. Enable and disable

Format the HDFS File System Using namenode

cd hadoop-0.20.2/confhadoop namenode -format

Enable hadoop

cd hadoop-0.20.2/binsh start-all.sh

Run the JPS command to check whether the expected hadoop process is running.

Disable hadoop

cd hadoop-0.20.2/binsh stop-all.sh

 

4. Reference

Running hadoop on Ubuntu Linux (single-node cluster)

Description of parameters in hadoop configuration files

Hadoop installation Configuration