[0010] Windows Eclipse Development HDFS Program sample (II)

Objective:

Learn the configuration of Windows development Hadoop programs

Related:

[0007] Example of an Eclipse development HDFs program under Windows

Environment:

Based on the following environment configuration is good.

[0008] Windows 7 under Hadoop 2.6.4 Eclipse Local Development Debug Configuration

1. New HDFs download File class

Add the following code to the new class in an existing MapReduce project, and the code removes the minor changes from [0007]

Features: Download files from HDFs to Windows Local

 PackageHadoop.hdfs;Importjava.io.FileNotFoundException;ImportJava.io.FileOutputStream;Importjava.io.IOException;Importorg.apache.commons.compress.utils.IOUtils;Importorg.apache.hadoop.conf.Configuration;ImportOrg.apache.hadoop.fs.FSDataInputStream;ImportOrg.apache.hadoop.fs.FileSystem;ImportOrg.apache.hadoop.fs.Path;/*** Function: Download hdfs://ssmaster:9000/data/paper.txt to Windows C:\paper.txt * Call method: Hadoop jar package name. Jar*/ Public classHdfs_download { Public Static voidMain (string[] args) {Configuration conf=NewConfiguration (); FileSystem FS=NULL; Path src=NULL; Fsdatainputstream in=NULL; FileOutputStream out=NULL; //src = new Path ("HDFs://ssmaster:9000/data/paper.txt ");src=NewPath ("Hdfs://ssmaster:9000/data/paper.txt" ); Try{FS=filesystem.get (conf); Inch=Fs.open (SRC); } Catch(IOException e) {e.printstacktrace (); }        Try{ out=NewFileOutputStream ("C:\\paper.txt"); } Catch(FileNotFoundException e) {e.printstacktrace (); }        Try{ioutils.copy (in, out); } Catch(IOException e) {e.printstacktrace (); }}}

View Code

2. Copy the Hadoop configuration core-site.xml,hdfs-site.xml,mapred-site.xml to the SRC directory

3 run

Right-click Class name, run as Java application, normal execution success

Abnormal

Refer to [0009]hadoop exception handling, which is generally not a configuration file

Summarize:

More convenient than [0007] package execution.

A problem exists:

If the package is executed under Linux, you need to modify the path

How to run the program directly in Eclipse, download HDFs to Linxu Local

[0010] Windows Eclipse Development HDFS Program sample (II)