Eclipse connectivity and use of Hadoop clusters on Linux in the win system

Preparatory work

First add the following to the hosts file of the win system:

10.61.6.164Master//hadoop cluster master node

First, install the Hadoop plugin on eclipse

Download Hadoop-eclipse-plugin-1.1.2.jar. Copy it to plugins under the Eclipse installation folder, and then start Eclispe. This is what you'll see in Eclipse's File/new/other, proving that the plugin was installed successfully

watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqvzgh0ef93emds/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/ Dissolve/70/gravity/center "style=" border:none; max-width:100% ">

Second, open the Map/reduce Locations in Window/show view/other

When you open it, you'll see the form. Click on the right side of the purple Elephant

The form you see will pop up.

Then configure your connection information:

Location name: Can be defined for whatever name

Host: Is the IP address of the master node of your Hadoop cluster

Port: Required by configuration, of course, if you configure the Hadoop cluster, the default port will be changed, here is your own change of the port number

You'll see it in the left-hand folder bar after the configuration.

Iii. Configuring program execution parameters (your project must be a MapReduce project, and you have added all the jar packages in Hadoop)

Create an in folder under your project, copy the data file into it, and then export your project to the jar file. Then add the following code to the main function of your project

Conf.set ("Mapred.jar", "E://freqitemset.jar");//mapred.jar cannot be changed

Right-click your project. Select Run As/run Configurations

watermark/2/text/ahr0cdovl2jsb2cuy3nkbi5uzxqvzgh0ef93emds/font/5a6l5l2t/fontsize/400/fill/i0jbqkfcma==/ Dissolve/70/gravity/center "style=" border:none; max-width:100% ">

Click Arguments

The content that is added inside

Lee file storage path on HDFs <dfs_path>

In/data input file (local path) <input>

3 Item set Size K

1 support level thresholds <spt_dg>

out output file < local path ><output>

Click OK and your project will be able to connect and use your Hadoop cluster.

Eclipse connectivity and use of Hadoop clusters on Linux in the win system