Configuring Hadoop2.6.0 in Eclipse

1. Download and configure the plugin

Place the downloaded Hadoop-eclipse-plugin-2.6.0.jar in the eclipse's Dropins directory and restart Eclipse.

2. Configure the Hadoop installation directory

• Windows->preference, Hadoop map/reduce, Hadoop installation directory

3. Configure Map/reduce View

• Window->open Perspective, other->map/reduce click "OK"
• Windows→show view→other->map/reduce locations-> Click "OK"
• The console will have one more tab page for "Map/reduce Locations"

On the "Map/reduce Locations" tab, click the icon < elephant +> or right click on the blank, select "New Hadoop location ...", and the dialog box "new Hadoop locations ..." pops up, Configure the following: Change HA1 to your own Hadoop user

Note: The MR master and DFS master configurations must be consistent with the configuration files such as Mapred-site.xml and Core-site.xml

Open Project Explorer to view the HDFs file system.

4. New Map/reduce Task

File->new->project->map/reduce Project->next

Write the WordCount class: Remember to get the service up first

[Java]View Plaincopy

2. Import java.io.IOException;
4. Import java.util.*;
6. Import Org.apache.hadoop.fs.Path;
8. Import org.apache.hadoop.conf.*;
10. Import org.apache.hadoop.io.*;
12. Import org.apache.hadoop.mapred.*;
14. Import org.apache.hadoop.util.*;
16. Public class WordCount {
18. Public static class Map extends Mapreducebase implements Mapper<longwritable, text, text, intwritable > {
20. Private final static intwritable one = new intwritable (1);
22. Private text word = new text ();
26. Public void Map (longwritable key, Text value, Outputcollector<text, intwritable> output, Reporter Reporter) throws IOException {
28. String line = value.tostring ();
30. StringTokenizer tokenizer = new StringTokenizer (line);
32. While (Tokenizer.hasmoretokens ()) {
34. Word.set (Tokenizer.nexttoken ());
36. Output.collect (Word, one);
38. }
40. }
42. }
44. Public static class Reduce extends Mapreducebase implements Reducer<text, Intwritable, Text, intwritable> {
46. Public void Reduce (Text key, iterator<intwritable> values, Outputcollector<text, intwritable> output, Reporter Reporter) throws IOException {
48. int sum = 0;
50. While (Values.hasnext ()) {
52. Sum + = Values.next (). get ();
54. }
56. Output.collect (Key, new intwritable (sum));
58. }
60. }
62. Public static void Main (string[] args) throws Exception {
64. jobconf conf = new jobconf (WordCount.   Class);
66. Conf.setjobname ("WordCount");
70. Conf.setoutputkeyclass (Text.   Class);
72. Conf.setoutputvalueclass (intwritable.   Class);
76. Conf.setmapperclass (Map.   Class);
78. Conf.setreducerclass (Reduce.   Class);
82. Conf.setinputformat (Textinputformat.   Class);
84. Conf.setoutputformat (Textoutputformat.   Class);
88. Fileinputformat.setinputpaths (conf, new Path (args[0]));
90. Fileoutputformat.setoutputpath (conf, new Path (args[1]));
94. Jobclient.runjob (conf);
96. }
98. }

5. Configure run-time parameters: Right-click-->run as-->run confiugrations

User/ha1/input/hadoop is the file you upload in the HDFs folder (which you create yourself) and put in the files you want to process. OUPUT4 Output Results

Run the program on a Hadoop cluster: Right--->runas-->run on Hadoop, and the final output will be displayed in the appropriate folder in HDFs. At this point, Ubuntu hadoop-2.6.0 Eclipse plugin configuration is complete.

Special Note: The Execute Mr Console does not have log details when configured as above.

Solution, copy the Hadoop-2.6.0/etc/haoop/log4j.properties file to the Eclipse project (for example, copy it to the/home/hadoop/workspace/wordcount path < Confusion: Copying to this path in the Linux file system is not valid, but copying from the Eclipse view is valid. and the corresponding generated log4j.properties file >) in the above path.

http://blog.csdn.net/zythy/article/details/17397153

Configuring Hadoop2.6.0 in Eclipse