Hadoop Tutorial (i) 1.2.1 true cluster installation

Experimental environment

192.168.56.2 Master.hadoop

192.168.56.3 Slave1.hadoop

192.168.56.4 Slave2.hadoop

One installation JDK

#/etc/profile

Export Java_home=/usr/local/java/default

Export path= $JAVA _home/bin: $JAVA _home/jre/bin: $PATH

Export classpath=.: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar

# Source/etc/profile

Two no password SSH login (recommended this machine also get)

# Local execution: Generate key pair

Ssh-keygen-t Dsa-p '-F ~/.SSH/ID_DSA

# Remote Server execution: Authorize the remote machine with the public key

# Note: On the Namenode, for the Secondarynamenode also need to operate a local no key login

Cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

# After remote machine authorization, you can delete the public key

RM-RF ~/.ssh/id_dsa.pub

Three configuration Hadoop

1) Configure conf/hadoop-env.sh

Export Java_home=/usr/local/java/default

2) Configure Conf/core-site.xml

If the Hadoop.tmp.dir parameter is not configured, the default temporary directory for the system is:/tmp/hadoo-hadoop. And this directory will be killed after each reboot, you must rerun the format to do, or else error

<configuration>

<property>

<name>hadoop.tmp.dir</name>

<value>/usr/hadoop/tmp</value>

<!--NOTE: Create a TMP folder in the/usr/hadoop directory first-->

<description>a base for other temporary directories.</description>

</property>

<!--file System Properties-->

<property>

<name>fs.default.name</name>

<value>hdfs://192.168.56.2:9000</value>

</property>

</configuration>

3) Configure Hdfs-site.xml

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

See more highlights of this column: http://www.bianceng.cnhttp://www.bianceng.cn/webkf/tools/

<!--NOTE: Replication is the number of copies of the data, for example: set to 3, salve less than 3 will be the error, the number of copies does not need to set too much-->

</property>

<configuration>

4) Configure Mapred-site.xml

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>192.168.56.2:9001</value>

</property>

</configuration>

5) Configure Masters (Secondarynamenode), use hosts

Master.hadoop

6) configuration slaves, Namenode unique Datanode can not be configured to use the hosts

Slave1.hadoop

Slave2.hadoop

7 other Datanode copy Namenode Hadoop can

Four start

Bin/hadoop Namenode-format

SH bin/start-all.sh

Verify the port, on the Namenode

http://192.168.56.2:50030/

http://192.168.56.2:50070/

Five extensibility

Direct copy Datanode to other machines while adding Namenode Masters

Note the hosts configuration of the new extension machine, if you use IP configuration you can not care about this

Attention:

Namenode must be configured with masters and slaves

Datanode only need to configure Masters, do not need slaves configuration

Other configurations are exactly the same

Remember to configure the password-free SSH login, or you will not start

Remember to perform the format command Bin/hadoop Namenode-format before starting, otherwise it will not start

Remember to configure the conf/hadoop-env.sh java_home before you start, otherwise you will not be able to start