Copy local files to the Hadoop File System

Copy local files to the Hadoop File System

// Copy the local file to the Hadoop File System
// Currently, other Hadoop file systems do not call the progress () method when writing files.
Package com;

Import java. io. BufferedInputStream;
Import java. io. FileInputStream;
Import java. io. InputStream;
Import java. io. OutputStream;
Import java.net. URI;

Import org. apache. hadoop. conf. Configuration;
Import org. apache. hadoop. fs. FileSystem;
Import org. apache. hadoop. fs. Path;
Import org. apache. hadoop. io. IOUtils;
Import org. apache. hadoop. util. Progressable;

Public class FileCopyWithProgress {
Public static void main (String [] args) throws Exception {
String localSrc = args [0];
String dst = args [1];
InputStream in = new BufferedInputStream (new FileInputStream (localSrc ));
Configuration conf = new Configuration ();
FileSystem fs = FileSystem. get (URI. create (dst), conf );
OutputStream out = fs. create (new Path (dst), new Progressable (){
Public void progress (){
System. out. println (".");
}
});

IOUtils. copyBytes (in, out, 4096, true );

}
}

Tutorial on standalone/pseudo-distributed installation and configuration of Hadoop2.4.1 under Ubuntu14.04

Install and configure Hadoop2.2.0 on CentOS

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)