Eclipse connectivity and use of Hadoop clusters on Linux in the win system

Preparatory work

Add the following to the hosts file of the win system first:

10.61.6.164Master//hadoop cluster master node

First, install the Hadoop plugin on eclipse

Download the Hadoop-eclipse-plugin-1.1.2.jar, copy it to plugins in the Eclipse installation directory, and start Eclispe, and you'll see something under Eclipse file/new/other. Prove plug-in installation successful

Second, open the Map/reduce Locations in Window/show view/other

Open the window that will show you, click on the purple elephant on the right

The window shown will pop up.

Then configure your connection information:

Location Name: You can define yourself as any name

Host: Is the IP address of the master node of your Hadoop cluster

Port: Required by configuration, of course, if you configure the Hadoop cluster to change the default port, here is your own modified port number

After configuration, you will see it in the left directory.

Third, configure the program running parameters (your project must be a MapReduce project, and have added all the jar packages in Hadoop)

First create an in directory under your project, copy the data file into it, export your project to the jar file, and add the following code to your project's main function

Conf.set ("Mapred.jar", "E://freqitemset.jar");//mapred.jar cannot be changed

Right-click on your project and select Run as/run configurations

Click Arguments

Add content from inside

Lee file storage path on HDFs <dfs_path>

In/data input file (local path) <input>

3 Item set Size K

1 support level thresholds <spt_dg>

out output file < local path ><output>

Click OK to connect and use your Hadoop cluster with your project

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Eclipse connectivity and use of Hadoop clusters on Linux in the win system