Working with hadoop under eclipse

1. eclipse3.6 SVN plugin
Installation --- subversive, my environment is ubuntu10.10, eclipse 3.6.2 helio. The installation steps are as follows:

Eclipse --> 'help' -- 'Install new
Software'

The Subversive URL is already built in Helio, but it is disable.

Select 'available
Software sites, then filter the subversive, and give the subversive URL to enable. Then return to the installation interface, you can select the subversive plug-in. After the installation is complete, restart.

2. After restarting, open the svn repositories view. Because no connector is installed, a window is automatically displayed, listing subversive connector ctor plugin, svnkit, and javahl. I chose the highest version of svnkit. Install and restart.

3. Download hadoopSource code

Hadoop SourceCodeIt consists of three parts: Common, HDFS (Distributed File System), and mapred (mapreduce). Their Respective SVN addresses are as follows:

Http://svn.apache.org/repos/asf/hadoop/common

Http://svn.apache.org/repos/asf/hadoop/hdfs

Http://svn.apache.org/repos/asf/hadoop/mapreduce

[Note] Three projects have dependencies. The common component is the public component of the latter two, while the mapred component is based on HDFS. However, these dependencies have been well processed without developers' concern, three projects can be separately downloaded and compiled
. /Trunk must be added after the download path.

*
In eclipse, click File> Import, select SVN> check items from SVN, and click Next to go to the next step;

*
Select "Create a new resource library location", click the next button, enter the address of the corresponding part, and then only select the trunk folder;

*
In the next step, select "check out as a new project and configure using the new Project Wizard". Do not select "check out as a project in the workspace" because the project type cannot be selected; click Finish to enter the new project wizard;

*
Select Java project as the project type, and select the project name. JRE should select 1.6. Click Next to enter the Java setting window;

*
Change the default Output Folder from.../bin
.../Build/eclipse-classes (or other) to prevent eclipse from automatically clearing the manually written files in the bin folder. Click Finish to wait for the download to complete.

At this time, there will be a lot of files marked with a Red Cross number, which is a normal phenomenon. Later we will use ant to compile the supporting Eclipse project file to eliminate these errors.

4. Set to compile using ant Builder

Taking mapreduce as an example, set the project created in the previous step as mapreduce.

*
Right-click the project (or select Properties from the project menu), find the builders tab, deselect the Java builder check box, and click the new button to select ant builder to enter the edit Configuration window;

*
It is initially the main tab. You can set the name mapcece_builder (or other) in the name column. In the buildfile column, select the build. xml file under the mapreduce project through the Browse workspace button;

*
Switch to the targets tab, and click the set targets button in the manual build column. In addition to the default compile options, select compile-test-system andEclipse (if not selected, the package directory structure does not match the Java file. For details, refer to)(Automatically generate the Eclipse project file). If the search is inconvenient, select sort targets (in alphabetical order of names) and hide internal targets.

After confirmation, the project starts compilation (For A Long Time). You can see that the console has various outputs, and several files will be automatically downloaded during this period. Right-click the project and select refresh (or press F5). You can see that the organization structure of the project has changed and the old Red Cross has disappeared.

[Note] due to the large project scale, after compiling on the first side, you can deselect build automatically in the project menu.

[Reference]

A. hadoop project SVN address:

Http://svn.apache.org/repos/asf/hadoop/

B. hadoop eclipse environment Setup Guide:

Http://wiki.apache.org/hadoop/EclipseEnvironment