(4) Implement local file upload to Hadoop file system by calling Hadoop Java API

(1) First create Java project

Select File->new->java Project on the Eclipse menu.

and is named UploadFile.

(2) Add the necessary Hadoop jar packages

Right-click the JRE System Library and select Configure build path under Build path.

Then select Add External Jars. Add the jar package and all the jar packages under Lib to your extracted Hadoop source directory.

All jar packages in the Lib directory.

(3) Join the UploadFile class

The code is as follows:

import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Progressable;


public class UploadFile {

	public static void main(String[] args) {
		
		try {
			String localSrc = "C://Goagent.rar";
			  String dst = "hdfs://hadoop:9000/user/root/Goagent.rar";
			  InputStream in = new BufferedInputStream(new FileInputStream(localSrc));
			  Configuration conf = new Configuration();
			  
			  FileSystem fs = FileSystem.get(URI.create(dst), conf);
			  OutputStream out = fs.create(new Path(dst), new Progressable() {
			   public void progress() {
			    System.out.print(".");
			   }
			  });
			  IOUtils.copyBytes(in, out, 4096, true);
			  System.out.println("success");
			
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
	}

}

Then execute the program, assuming that the upload succeeds in the output success under the console.

You can also view http://hadoop:50070/on the Web page.

Note: I am also just starting to learn Hadoop. There may be some parts of the article that are incomplete or wrong. We also invite you to enlighten us, but also hope to exchange learning. promote each other to improve.

References:

http://my.oschina.net/cuitongliang/blog/155954

(4) Implement local file upload to Hadoop file system by calling Hadoop Java API