Simple installation deployment process for Hadoop

simple installation deployment process for Hadoop

In order to do some experiments, so on their own laptop installed a virtual machine, the system for centos6.2,jdk1.7,hadoop-1.0.1

For the sake of simplicity, deploying pseudo-distributed, that is, only one node, this node is both master and slave, both Namenode and Datanode, both Jobtracker and Tasktracker.

Deployment General Description:

Pseudo-distributed deployment is relatively simple, only need to solve 4 configuration files, respectively:

1.hadoop-env.sh// to specify the location of the JDK

2.core-site.xml// core configuration to specify the HDFs address and port number

3.hdfs-site.xml//hdfs configuration, you can specify the number of backups, default is 3, pseudo-distributed needs to be configured to 1

4.mapred-site.xml// address and port used to configure Jbotracker

After configuring the above files, there are two steps ahead:

1. Format the HDFs file system

2. start and verify

Official start:

1. Configure hadoop-env.sh

Having forgotten the location of the JDK, I checked with java-verbose and found/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.85.x86_64/jre

Therefore, write the following line in the hadoop-env.sh (in fact, you can find the specified location, the reader can find their own)

Export JAVA_HOME=/USR/LIB/JVM/JAVA-1.7.0-OPENJDK-1.7.0.85.X86_64/JRE

2. Configure Core-site.xml

<?xmlversion= "1.0"?>

<?xml-stylesheettype= "text/xsl" href= "configuration.xsl"?>

<!--Putsite-specific Property overrides the This file. -

<configuration>

<property>

<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

// Description: In fact, the other parts have already, just need to tap the black and bold part of their own can be

3. Configure Hdfs-site.xml

<?xmlversion= "1.0"?>

<?xml-stylesheettype= "text/xsl" href= "configuration.xsl"?>

<!--Putsite-specific Property overrides the This file. -

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

</configuration>

4. Configure Mapred-site.xml

<?xmlversion= "1.0"?>

<?xml-stylesheettype= "text/xsl" href= "configuration.xsl"?>

<!--Putsite-specific Property overrides the This file. -

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>localhost:9001</value>

</property>

</configuration>

Go to the last two steps:

1. format HDFs

[email protected] hadoop]# cd/usr/local/hadoop/bin// Enter the executable file bin directory for Hadoop

[email protected] bin]# /hadoop namenode-format// execute a format command

2. start and verify

[Email protected] bin]# ./start-all.sh

Open the browser verification, enter the following URL, respectively:

http://localhost:50030 (Web page for MapReduce)

http://localhost:50070 (HDFs Web page)

---- Okay, OK.

Simple installation deployment process for Hadoop