Hadoop entry (1): hadoop pseudo distribution Installation

 

1. Install hadoop

First, extract the downloaded hadoop 0.20 package to the/home/Admin directory:

Tar xzfhadoop-0.20.2.tar.gz

 

Configure hadoop environment variables:

Exporthadoop_install =/home/admin/hadoop-0.20.2

Exportpath = $ path: $ hadoop_install/bin

 

Test whether the installation is successful:

Hadoop version

2. Create an SSH password-less key

Configure SSH logon without a password for the current user:

Ssh-keygen-t rsa-p'-f ~ /. Ssh/id_rsa

Cat ~ /. Ssh/id_rsa.pub> ~ /. Ssh/authorized_keys

 

Test whether the system prompts you to enter the password:

SSH localhost

3. Configure the hadoop pseudo Distribution Mode

/Home/admin/hadoop-0.20.2/CONF/core-site.xml

========================================================== ========================================================

<Configuration>

<Property>

<Name> fs. Default. Name </Name>

<Value> HDFS: // localhost </value>

</Property>

</Configuration>

 

/Home/admin/hadoop-0.20.2/CONF/hdfs-site.xml

========================================================== ========================================================

<Configuration>

<Property>

<Name> DFS. Replication </Name>

<Value> 1 </value>

</Property>

</Configuration>

 

/Home/admin/hadoop-0.20.2/CONF/mapred-site.xml

========================================================== ========================================================

<Configuration>

<Property>

<Name> mapred. Job. Tracker </Name>

<Value> localhost: 8021 </value>

</Property>

</Configuration>

4. Start hadoop service 4.1 to format namenode

Hadoop namenode-format

4.2 start the service

Start-dfs.sh

Start-mapred.sh

4.3 FAQs

Datanode error found when namenode STARTUP script % hadoop_home %/bin/start-dfs.sh:

Error: java_homeis not set

The reason is that the definition of java_home is missing in % hadoop_home %/CONF/hadoop-env.sh, just add:

Exportjava_home =/export/servers/jdk1.6.0 _ 25/

5. Test HDFS

Hadoop FS-mkdir books

Hadoop FS-ls.

Hadoop FS-copyfromlocal notice.txt HDFS: // localhost/user/root/books/notice.txt

 

References

Hadoop authoritative guide