hadoop2.x Eclipse under Programming environment configuration

This process is for how to write hadoop2.x corresponding mapreduce programs in Eclipse and then package them and upload them to the Hadoop cluster execution process.

1. Download the hadoop2.x tar package and extract it to a directory ( Note: Not a source package, it is an installation package )

2. Import the Hadoop2x-eclipse plugin into eclipse:

(1) Compile the plugin. Plugin Source Address Https://github.com/winghc/hadoop2x-eclipse-plugin

(2) Copy the compiled Hadoop2x-eclipse*.jar package to the Eclipse/plugins directory

(3) Restart Eclipse

(4) Eclipse-> windows-> Open Perspective opens the MapReduce view

(5) eclipse-> windows->preferences-> Hadoop MapReduce, configure the Hadoop installation path, point to 1 in the directory after extracting the Hadoop tar package.

3. Create a MapReduce project

(1) Create project: eclipse-> file-> new-> map/reduce project. After the project is created, a series of dependent jar packages, including hadoop-hdfs*, hadoop-yarn* and so on, are found in the project directory. If not, then write the code will be error, you need to manually import.

A series of jar packages that you will see in the project:

(2) You can then start the development. The map or reduce file created by new-> Mapper/reducer/driver in the project will automatically generate references, inheritance, and structure, and only need to add the implementation.

The structure has automatically generated a good map file:

4. Package, upload to cluster, run

(1) eclipse-> file-> export-> Jar File. Export JAR Package

(2) Upload the jar package to the cluster and run:

Hadoop jar Wordcount.jar org.apache.hadoop.wordcount arg0 arg1 ...

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

hadoop2.x Eclipse under Programming environment configuration