(4) Implement local file upload to Hadoop file system by calling Hadoop Java API

Source: Internet
Author: User
Tags goagent


(1) First create Java project


Select File->new->java Project on the Eclipse menu.









and is named UploadFile.














(2) Add the necessary Hadoop jar packages


Right-click the JRE System Library and select Configure build path under Build path.









Then select Add External Jars. Add the jar package and all the jar packages under Lib to your extracted Hadoop source directory.









All jar packages in the Lib directory.






(3) Join the UploadFile class





The code is as follows:





import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.util.Progressable;


public class UploadFile {

	public static void main(String[] args) {
		
		try {
			String localSrc = "C://Goagent.rar";
			  String dst = "hdfs://hadoop:9000/user/root/Goagent.rar";
			  InputStream in = new BufferedInputStream(new FileInputStream(localSrc));
			  Configuration conf = new Configuration();
			  
			  FileSystem fs = FileSystem.get(URI.create(dst), conf);
			  OutputStream out = fs.create(new Path(dst), new Progressable() {
			   public void progress() {
			    System.out.print(".");
			   }
			  });
			  IOUtils.copyBytes(in, out, 4096, true);
			  System.out.println("success");
			
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
	}

}





Then execute the program, assuming that the upload succeeds in the output success under the console.














You can also view http://hadoop:50070/on the Web page.












Note: I am also just starting to learn Hadoop. There may be some parts of the article that are incomplete or wrong. We also invite you to enlighten us, but also hope to exchange learning. promote each other to improve.






References:



http://my.oschina.net/cuitongliang/blog/155954















(4) Implement local file upload to Hadoop file system by calling Hadoop Java API


Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.