Install hadoop on centos and run the wordcount instance

Reference http://hadoop.apache.org/docs/r0.19.2/cn/index.html for the entire configuration process

The Linux system is centos6.2.

1. Install JDK. Download the latest jdk rpm package from Oracle and double-click it. The system is installed in the/usr/Java/jdk1.7.0 _ 07 folder by default.

2. download the latest stable version hadoopin apache. my next is hadoop-1.0.3.tar.gz.

3. Unzip to the folder hadoop-1.0.3 folder and configure the document (here I am configuring the pseudo distribution mode directly ).

3.1 hadoop-1.0.3/CONF/hadoop-env.sh configuration java_home for JDK installation path: Export java_home =/usr/Java/jdk1.7.0 _ 07

3.2 hadoop-1.0.3/CONF/core-site.xml add

<property>    <name>fs.default.name</name>    <value>hdfs://localhost:9000/</value></property>

3.3 hadoop-1.0.3/CONF/mapred-site.xml add

<property>    <name>mapred.job.tracker</name>    <value>localhost:9001</value></property>

3.3 hadoop-1.0.3/CONF/hdfs-site.xml add

<property>    <name>dfs.replication</name>    <value>1</value></property>

4. Then run bin/hadoop version in the hadoop-1.0.3 directory. If the output is correct to the version, the hadool configuration is successful.

5. Install SSH: Yum Install SSH. When I run this command, I am prompted that the system has installed SSH and is the latest version.

6. log on to SSH and set password-free settings. If the SSH localhost cannot be directly logged on, run the following command to log on directly.

Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa
Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys

7. Format namenode. Run bin/hadoop namenode-format

8. Start hadoop. Run bin/start-all.sh

9. Run the wordcount example. First, put the input file required for the example on DFS, but an error occurs when bin/hadoop FS-Put/usr/input/home/input is executed. It is found that datanode is not started. The method found on the internet is that the default hadoop. TMP. dir path:/tmp/hadoop-$ {user. name}, while the type of the/tmp directory file system in my Linux system is often not supported by hadoop. So here we need to change the path of hadoop. tmp. dir to another place. Here we need to specify a sub-directory under the hadoop installation path.

10. Stop hadoop, execute the bin/stop-all.sh, and then add in the core-site.xml:

<property>    <name>hadoop.tmp.dir</name>    <value>/home/puweilan/BigData/hadooptmp</value></property>

11. Format namenode again, start hadoop, and execute bin/hadoop FS-Put/usr/input/home/input.

12. For detailed steps to run wordcount, see http://hadoop.apache.org/docs/r0.19.2/cn/mapred_tutorial.html