Running Hadoop programs under Eclipse

Environment: Ubuntu installed on virtual machines, Hadoop installed in Ubuntu,eclipse installed under Win7, so eclipse will not be too card.

1. Copy the Eclipse plugin jar package for the corresponding version of Hadoop, such as my Hadoop is 0.21.0, and use the Hadoop-0.21.0-eclipse-plugin.jar,jar package in the Eclipse installation directory/plugins /Down, restart Eclipse

2. Add Map/reduce

3, add the Hadoop path, this does not add Ubuntu inside, directly add Win7 under the extracted files can be, this path is just to load the jar package

4, Configuration Map/reduce Locations

LocationName, Map/reduce Master and DFS master, the host and port are the addresses and ports you configured in Mapred-site.xml, Core-site.xml, respectively. Such as

Under adwanced parameters find Hadoop.tmp.dir, fill in the corresponding address in Core-site.xml

Click Finish to see the folder in the upper left corner (if you open the folder error, the Hadoop plug-in and Hadoop version does not match, or the Eclipse version does not match, replace the eclipse version of the line)

5. New Map/reduce Project

File-->new-->other-->map/reduce Project
The project name can be taken casually, such as hadoop-test.
Copy the Hadoop installation directory/src/example/org/apache/hadoop/example/wordcount.java to the project you just created.

6, upload the simulation Data folder.

In order to run the program, we need an input folder, and an output folder. The output folder does not need to be created and will be generated automatically when the program finishes running. We need to give the program an input folder.

In the current directory (such as the Hadoop installation directory), create a new folder input, and under the folder new two files F_1, f_2, the content is arbitrarily added.

7, run the project.

Add under Arguments

Hdfs://192.168.136.144:9000/user/zhouyao/input HDFS://192.168.136.144:9000/USER/ZHOUYAO/OUTPUT1

These two parameters represent the Namenode address, the input file address, the output file address

Click Run to see the following information

Under Folders, open output to see the results of the run