Ubuntu 14.10 under Eclipse install Hadoop plugin

Prepare the Environment

1 installed Hadoop, previously installed Hadoop 2.5.0, installation reference http://www.cnblogs.com/liuchangchun/p/4097286.html

2 install Eclipse, which can be downloaded directly from its website

Installation steps

1 Download Eclipse plugin, I am looking for a Hadoop 2.2 plug-in, in Hadoop 2.5 can be used normally, get plug-ins here are two ways

1.1 One is to download their own source code compiled, the process is as follows

First, download Eclipse-hadoop plugin, URL is https://github.com/winghc/hadoop2x-eclipse-plugin, you can click on the bottom right of the page download Zip download. After downloading, unzip,.

Then, go to the Hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin folder and execute the command.

Ant jar-declipse.home=/usr/local/eclipse-dhadoop.home=~/downloads/hadoop-2.2.0-dversion=2.5.0

The build goes through, and the generated plug-in is in the Hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin directory.

1.2 or download the compiled plugin directly,http://pan.baidu.com/s/1mgiHFok

　　2 Copy the downloaded plugin to the Eclipse/plugins directory and need to restart Eclipse

3 Configuring Hadoop installation directory

3.1 If the plugin is successfully installed, after opening windows-preferences, there will be a Hadoop map/reduce option on the left side of the window, click this option to set the Hadoop installation path on the right side of the window.

3.2 configuration map/reduce Locations: open windows-open perspective-other Select Map/reduce, click OK

3.3 Tap the Map/reduce Location tab and click on the icon on the right to open the Hadoop location Configuration window: Enter location name, any name. Configure Map/reduce Master and DFS Mastrer,host and port to be configured to match core-site.xml settings. If you do not modify the port yourself, then one is 9001, one is 9000

3.4 Click on the dfslocations-> locationnameon the left (location name in the previous step), if you can see the file under Hadoop, the installation is successful.

4 Test MapReduce. In Eclipse,file-> Project, select Map/reduce Project, enter the name WordCount, and so on. Then create a new class, under the code copy

Importjava.io.IOException;ImportJava.util.StringTokenizer;Importorg.apache.hadoop.conf.Configuration;ImportOrg.apache.hadoop.fs.Path;Importorg.apache.hadoop.io.IntWritable;ImportOrg.apache.hadoop.io.Text;ImportOrg.apache.hadoop.mapreduce.Job;ImportOrg.apache.hadoop.mapreduce.Mapper;ImportOrg.apache.hadoop.mapreduce.Reducer;ImportOrg.apache.hadoop.mapreduce.lib.input.FileInputFormat;ImportOrg.apache.hadoop.mapreduce.lib.output.FileOutputFormat;ImportOrg.apache.hadoop.util.GenericOptionsParser; Public classWordCount { Public Static classTokenizermapperextendsMapper<object, text, text, intwritable> {        Private Final StaticIntwritable one =NewIntwritable (1); PrivateText Word =NewText ();  Public voidmap (Object key, Text value, context context)throwsIOException, interruptedexception {stringtokenizer ITR=NewStringTokenizer (value.tostring ());  while(Itr.hasmoretokens ()) {Word.set (Itr.nexttoken ());            Context.write (Word, one); }        }    }     Public Static classIntsumreducerextendsReducer<text, Intwritable, Text, intwritable> {        Privateintwritable result =Newintwritable ();  Public voidReduce (Text key, iterable<intwritable>values, context context)throwsIOException, interruptedexception {intsum = 0;  for(intwritable val:values) {sum+=Val.get ();            } result.set (sum);        Context.write (key, result); }    }     Public Static voidMain (string[] args)throwsException {Configuration conf=NewConfiguration (); String[] Otherargs=Newgenericoptionsparser (conf, args). Getremainingargs (); if(Otherargs.length! = 2) {System.err.println ("Usage:wordcount <in> <out>"); System.exit (2); } Job Job=NewJob (conf, "word count"); Job.setjarbyclass (WordCount.class); Job.setmapperclass (tokenizermapper.class); Job.setcombinerclass (intsumreducer.class); Job.setreducerclass (intsumreducer.class); Job.setoutputkeyclass (Text.class); Job.setoutputvalueclass (intwritable.class); Fileinputformat.addinputpath (Job,NewPath (otherargs[0])); Fileoutputformat.setoutputpath (Job,NewPath (otherargs[1])); System.exit (Job.waitforcompletion (true) ? 0:1); }}

5 run the project, need to do some preparatory work first

5.1. Create a directory on HDFs input

Hadoop fs-mkdir Input

5.2. Random copy of local README.txt into HDFs input

Hadoop fs-copyfromlocal/usr/local/hadoop/readme.txt Input

5.3, click Wordcount.java, right click on Run As->run configurations, configure the run parameters, namely the input and output folder

Hdfs://localhost:9000/user/hadoop/input Hdfs://localhost:9000/user/hadoop/output

5.4 Note that the input directory output should not be established in Hadoop, or it will be an error

6 viewing results, you can see multiple directories directly under DFS Locations Refresh, with results

--------------------------------------------------------------------------------------------------------------- -------------------------

WordCount program above is written in a class, the specification is a map class, reduce class, mapredcuedriver separate building, low coupling

1 new Map/reduce Engineering WordCount.

2 Create a new Mapper.java, select File-->new-->mapper, enter the package name and the class name.

3 Create a new Reduccer.java, select File-->new-->reducer, enter the package name and the class name.

4 Establish Map/reduce Driver, select File-->new-->mapreduce Driver, enter the package name and class name.

5 run, same as above

　　

　　

Ubuntu 14.10 under Eclipse install Hadoop plugin