Remote Java client uploads files to Hdfs__java

Hadoop cluster Environment:

Three machines: Namenode0, Datanode1, Datanode2 operating system: Ubuntu 11.04 Server version Haddop: hadoop-0.20.2-cdh3u1 HBase version: hbase-0.90.4-cdh3u2 java version: jdk-6u29-linux-x64

Client machine: Note: Conf needs to add Core-site.xml configuration file, and its content is correct. Permissions problem: The machine in the cluster is running in a Hadoop account, my local machine is running on Lotus account, if upload to hdfs://namenode0:54310/usr/directory will fail, throw permission denied exception. Because owner of the/usr directory is Hadoop, uploading to the hdfs://namenode0:54310/usr/lotus/directory is successful because the Lotus directory owner is set to Lotus account.

<?xml version= "1.0"?> <?xml-stylesheet type= "text/xsl"
href= "configuration.xsl"?>

<!--put Site-specific property overrides in this file. -->

<configuration>
	<property>
		<name>fs.default.name</name>
		< value>hdfs://namenode0:54310</value>
	</property>
</configuration>

Code

Import Java.io.File;

Import Java.net.URI;
Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.FileStatus;
Import Org.apache.hadoop.fs.FileSystem;


Import Org.apache.hadoop.fs.Path;
	public class Hdfsupload {//hdfsupload Srcfilepath, Dstfilepath public static void Main (string[] args) throws Exception
			{if (Args.length < 2) {System.out.print ("command line:hdfsupload srcfilepath dstfilepath");
		Return
		} String Srcfilepath = Args[0];
		
		String Dstfilepath = args[1];
		Configuration conf = new Configuration ();
		Conf.addresource (New Path ("/home/lotus/work/hadoop-0.20.2-cdh3u1/conf/core-site.xml"));
		
		
		FileSystem HDFs = filesystem.get (conf);
		Path src = new path (Srcfilepath);
		Path DST = new path (Dstfilepath);
		System.out.print ("Start upload");
		Long starttime = System.currenttimemillis ();
		Hdfs.copyfromlocalfile (SRC, DST);
		Long endtime = System.currenttimemillis (); System.out.print ("Finish upload, Cost time:" + string.valueof (EndtIme-starttime)); }

}

Performance test: Hdfsupload/media/pc-book/dd/win2000.dd hdfs://namenode0:54310/usr/lotus/win2000.dd upload a 3.9G DD file, took 428 seconds, The second upload took 377 seconds to study after the performance problem.