Build Hadoop2.4.0 development environment under eclipse

First, install Eclipse

Download Eclipse, unzip the installation, e.g. install to/usr/local, i.e./usr/local/eclipse

4.3.1 version: Http://pan.baidu.com/s/1eQkpRgu

Ii. Installing the Hadoop plugin on eclipse

1. Download the Hadoop plugin

: Http://pan.baidu.com/s/1mgiHFok

This zip file contains the source code, we use the compiled jar can be extracted, the release folder in the Hadoop.eclipse-kepler-plugin-2.2.0.jar is a compiled plug-in.

2, put the plug-in into the Eclipse/plugins directory

3. Restart Eclipse, configure Hadoop installation directory

If the plug-in installation succeeds, after opening windows-preferences, there will be a Hadoop map/reduce option on the left side of the window, click this option to set the Hadoop installation path on the right side of the window.

4, Configuration Map/reduce Locations

Open Windows-open Perspective-other

Select Map/reduce, click OK

In the lower right, see as shown

Click on the Map/reduce Location tab and click on the icon on the right to open the Hadoop location Configuration window:

Enter location name, any name. Configure Map/reduce Master and DFS Mastrer,host and port to be configured to match core-site.xml settings.

Click the "Finish" button to close the window.

Click on the left Dfslocations->myhadoop (location name in the previous step), if you can see user, the installation is successful.

If the installation fails as shown, check to see if Hadoop is started and the eclipse is configured correctly.

Iii. New WordCount Project

File->project, select Map/reduce Project, enter the item name WordCount, and so on.

Create a new class in the WordCount project named WordCount with the following code:

Import Java.io.ioexception;import Java.util.StringTokenizer; Import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.fs.path;import Org.apache.hadoop.io.intwritable;import Org.apache.hadoop.io.text;import Org.apache.hadoop.mapreduce.job;import Org.apache.hadoop.mapreduce.mapper;import Org.apache.hadoop.mapreduce.reducer;import Org.apache.hadoop.mapreduce.lib.input.fileinputformat;import Org.apache.hadoop.mapreduce.lib.output.fileoutputformat;import Org.apache.hadoop.util.GenericOptionsParser; public class WordCount {public static class Tokenizermapper extends Mapper<object, text, text, intwritable>{Priv　　Ate final static intwritable one = new intwritable (1); 　　Private text Word = new text ();  public void Map (Object key, Text value, Context context) throws IOException, interruptedexception {stringtokenizer ITR　　　　　　= New StringTokenizer (value.tostring ());　　　　　　　　while (Itr.hasmoretokens ()) {Word.set (Itr.nexttoken ());　　　　　　Context.write (Word, one); }　　}}public static class Intsumreducer extends Reducer<text,intwritable,text,intwritable> {private intwritable result 　　= new Intwritable (); public void reduce (Text key, iterable<intwritable> Values,context Context) throws IOException,　　　　interruptedexception {int sum = 0;　　　　for (intwritable val:values) {sum + = Val.get ();　　　　} result.set (sum);　　Context.write (key, result);　　}} public static void Main (string[] args) throws Exception {Configuration conf = new configuration ();　　string[] Otherargs = new Genericoptionsparser (conf, args). Getremainingargs ();　　　　if (otherargs.length! = 2) {System.err.println ("Usage:wordcount <in> <out>");　　System.exit (2);　　Job Job = new Job (conf, "word count");　　Job.setjarbyclass (Wordcount.class);　　Job.setmapperclass (Tokenizermapper.class);　　Job.setcombinerclass (Intsumreducer.class);　　Job.setreducerclass (Intsumreducer.class);　　Job.setoutputkeyclass (Text.class);　　Job.setoutputvalueclass (Intwritable.class); FileInputFormat.addinputpath (Job, New Path (Otherargs[0]));　　Fileoutputformat.setoutputpath (Job, New Path (Otherargs[1])); System.exit (Job.waitforcompletion (true)? 0:1);}}

Four, the operation

1. Create a directory on HDFS input

Hadoop Fs-mkdir Input

2. Copy the local README.txt to the input in HDFs

Hadoop fs-copyfromlocal/usr/local/hadoop/readme.txt Input

3, click Wordcount.java, right click on Run As->run configurations, configure the run parameters, namely the input and output folder

Hdfs://localhost:9000/user/hadoop/input Hdfs://localhost:9000/user/hadoop/output

Click the Run button to run the program.

4, after the completion of the operation, view the results of the operation

Method 1:

Hadoop fs-ls Output

You can see that there are two output results, _success and part-r-00000

Perform Hadoop fs-cat output/*

Method 2:

Expand Dfs Locations, as shown, double-click Open part-r00000 View Results

(turn) Eclipse to build the Hadoop2.4.0 development environment