Use eclipse to write hadoop applications

Here, a simple program is compiled using eclipse.

1) create a Java project dfsoperator under eclipse and create corresponding classes. Do not create a package by yourself when creating a class. Use the default package. It seems that you have created your own package and cannot find the class. Later, the class is re-created in the default package and runs successfully.

2) Configure compilation Parameters

Configure "build path", select "dfsoperator", right-click, and choose "build path-> configurebuild path" from the menu to go to the "Java build path" configuration page. Select the libraries tab and click the Add external jar button on the right to add the installed hadoop-0.20.2-core.jar.

3) EDIT source code

4) compile and generate a jar package

Right-click the "dfsoperator" project, select "build project", compile the project, compile "dfsoperator. Java", and generate the dfsoperator. Class file shown below:

Right-click the project, select "Export", and export "JAR file", that is, the jar package. Then upload the exported JAR file to the hadoop master node. I don't know how to upload the file to the hadoop master, So I skipped this and copied the JAR file directly to the bin directory and executed 5). The test was successful. But I still don't know what it means.

5) Run

Put the exported JAR file under the hadoop binfile, go to the hadoop bin directory, and run the following command for testing:

(1) ls: Check the current directory and check whether dfs_operator.jar exists;

(2)./hadoop FS-ls/: The dfs_operator.txt file exists in the hadooproot directory;

(3)./hadoop jar./dfs_operator.jar dfsoperator: Run dfs_operator.jarto generate the dfs_operator.txt file;

(4)./hadoop FS-ls/: add the dfs_operator.txt file to the hadooproot directory;

(5)./hadoop FS-CAT/dfs_operator.txt: Check the content of the dfs_operator.txt file.

This program provides simple functions, but you can learn how to use eclipse for hadoop programming in windows.