Using Eclipse to develop Hadoop applications under Linux

In a previous article, it was described that if you build Hadoop0.20.2 in a fully distributed environment, you can now use this environment to complete the development.

First use the Hadoop user to login to the Linux system (Hadoop users created in the previous article), and then download the Eclipse tar.gz package to/home/hadoop/this directory, the direct decompression, so there will be/home/hadoop/ Eclipse this directory. Before development, you need to copy a jar underneath the hadoop0.20.2 directory to eclipse. (Note: In the previous article I changed the hadoop0.20.2 folder into Hadoop, Haha, so the following directory do not faint Oh!) ）

First step: Plugin replication

First find the jar of Eclipse-plugin below Hadoop, located in/home/hadoop/hadoop/contrib/eclipse-plugin/ Hadoop-0.20.2-eclipse-plugin.jar

Assign it to the Eclipse directory you just unzipped, directory in/home/hadoop/eclipse/plugins/, open this directory, paste it.

Once the jar package is copied, you can double-click the Eclipse icon in the Eclipse directory to open Eclipse for development.

Step Two: Prepare the development environment

after opening eclipse, first windows->preferences, open as the interface, according to the Red Box prompt area to configure, where Hadoop is installed in the location of Hadoop download after the decompression.

Then again, Windows->show View->other, enter the map in the open page to find Map/reduce Locations, as

When you click OK, the Map/reduce Locations is displayed below the Eclipse development tool and right-click in the area to create a new Hadoop location so that we can manage the HDFs directory.

Click "New Hadoop location" will pop up a configuration page, where the parameters configured as in the identity, click Finish, you can appear on the left side of the development environment a Hadoop directory navigation tree.

Files in the HDFs directory in Hadoop can be found under the user folder. At this point, the basic environment configuration has been completed, the following can be formally developed.

Step three: Java code writing

Below we briefly elaborated, to complete the function, in the test_2.txt we recorded some cell phone number at different time calls, some is to call 10086, some is to call 120, and some other numbers, now we need to complete the work is statistics, such as 10086 by which number hit , there are 120 of the number hit, so, our expectations and target data are as follows:

First you need to create a new project: File->new->project ... In the pop-up form, select Map/reduce Project, click Next, enter Project Name here is MyHadoop002, then click Finish, you can see the new project on the left side of the development environment, Expand the newly created project MyHadoop002, select SRC, right-click, create a new class, enter name in the popup page (I entered here is Test_2) and tap Finish (package optional).

The code inside the Test_2.java is as follows:

Fourth step: Run the program

The code is finished, the last step is both run, click Run->run configurations, pop-up page, the left is the navigation, the right is the parameter configuration interface, first click on the coordinate navigation in the Java application, click "Add" icon, will be in the Java Application below generates a application with the same name as class name, here is test_2.

At this point, you need to configure the Java code in the main function of the two parameters, that is, the document read path, and reduce the output path of the document, in the main function we have to args[0], args[1], so, here we need to enter the HDFs path.

Click "Run" after the input is complete, if there is no problem, it will run normally, let us look at the results of the operation

In this case, a simple Linux program that leverages eclipse to develop Hadoop is complete. I hope you can learn from it ....

Using Eclipse to develop Hadoop applications under Linux