Objective:
Learn the configuration of Windows development Hadoop programs
Related:
[0007] Example of an Eclipse development HDFs program under Windows
Environment:
Based on the following environment configuration is good.
[0008] Windows 7 under Hadoop 2.6.4 Eclipse Local Development Debug Configuration
1. New HDFs download File class
Add the following code to the new class in an existing MapReduce project, and the code removes the minor changes from [0007]
Features: Download files from HDFs to Windows Local
PackageHadoop.hdfs;Importjava.io.FileNotFoundException;ImportJava.io.FileOutputStream;Importjava.io.IOException;Importorg.apache.commons.compress.utils.IOUtils;Importorg.apache.hadoop.conf.Configuration;ImportOrg.apache.hadoop.fs.FSDataInputStream;ImportOrg.apache.hadoop.fs.FileSystem;ImportOrg.apache.hadoop.fs.Path;/*** Function: Download hdfs://ssmaster:9000/data/paper.txt to Windows C:\paper.txt * Call method: Hadoop jar package name. Jar*/ Public classHdfs_download { Public Static voidMain (string[] args) {Configuration conf=NewConfiguration (); FileSystem FS=NULL; Path src=NULL; Fsdatainputstream in=NULL; FileOutputStream out=NULL; //src = new Path ("HDFs://ssmaster:9000/data/paper.txt ");src=NewPath ("Hdfs://ssmaster:9000/data/paper.txt" ); Try{FS=filesystem.get (conf); Inch=Fs.open (SRC); } Catch(IOException e) {e.printstacktrace (); } Try{ out=NewFileOutputStream ("C:\\paper.txt"); } Catch(FileNotFoundException e) {e.printstacktrace (); } Try{ioutils.copy (in, out); } Catch(IOException e) {e.printstacktrace (); }}}
View Code
2. Copy the Hadoop configuration core-site.xml,hdfs-site.xml,mapred-site.xml to the SRC directory
3 run
Right-click Class name, run as Java application, normal execution success
Abnormal
Refer to [0009]hadoop exception handling, which is generally not a configuration file
Summarize:
More convenient than [0007] package execution.
A problem exists:
If the package is executed under Linux, you need to modify the path
How to run the program directly in Eclipse, download HDFs to Linxu Local
[0010] Windows Eclipse Development HDFS Program sample (II)