Developing a mapreduce program in eclipse

I. Installation and setup of Eclipse

1. Download the eclipse-jee-oxygen-3a-linux-gtk-x86_64.tar.gz file on the Eclipse official website and copy it to/home/jun/resources, and then copy the file to/home/ June and unzip.

CP /home/jun/resources/eclipse-jee-oxygen-3a-linux-gtk-x86_64. tar. gz/home/jun/tar -zxvf/home/jun/eclipse-jee-oxygen-3a-linux-gtk-x86_64. Tar

2. Execute the. Eclipse program to start eclipse

[Email protected] ~]$ CD eclipse/lsartifacts.xml  configuration  dropins  eclipse  Eclipse.ini  features  icon.xpm  p2  plugins  readme[[email protected] eclipse]$.

3. Installing the Hadoop plugin

(1) Download: Https://github.com/winghc/hadoop2x-eclipse-plugin/blob/master/release/hadoop-eclipse-plugin-2.6.0.jar

(2) Put the Hadoop-eclipse-plugin-2.6.0.jar under the/home/jun/eclipse/plugins

(3) Start Eclipse, select Window→preferences→hadoop map/reduce→hadoop installation directory→ Select/home/jun/hadoop

(4) Select Window→show view→other→mapreduce tools→map/reduce locations→ok

(5) in the lower Map/reduce locations Sub-window, right-click-new Hadoop Location

(6) Press to configure, click Finish

　　

(7) See the picture below on the left

　　

　　

Ii. developing a mapreduce program on eclipse

1. Create a new input.txt and upload it to the/test folder in HDFs and right-click Refresh in eclipse to see the file you just uploaded. In eclipse, you can do things like file directory creation, file uploads, file downloads, file or folder deletions, but you can't edit the contents of a file.

　　

2. New Project

Select File→new→other→map/reduce Project→next→project name{wordcount}→next→finish

Then create a new package and class

3. Access to source code

Go to the desktop

Select Hadoop-mapreduce-examples-2.8.4-sources.jar and right-click Open with Archive Manager, find Wordcount.java with the gedit opened to get the source code

Then copy the source code into Eclipse's new class

4. Running the program

Right-click on the Java file to Run as→run Configurations→java application→arguments

The first parameter is the input file, the second parameter is the output directory (it must not exist before), and then click Apply

　　

Right-click on the Java file on the Run as→run on Hadoop

The output subdirectory is then added to the Hadoop test directory with two files below, where part-r-000000 contains the results of the calculation

　　

Here, the first one to develop and run the MapReduce program on Eclipse is completed.

Developing a mapreduce program in eclipse