Some basic operations for Hadoop

Here are some of the basic Hadoop file operations, mainly upload files, download files and delete files, but to use the following features need to pay attention to some places:

1, to note whether the Windows system has been configured with the domain name and IP mapping relationship;

2, in order to operate the file on the HDFS system, you can use "/" instead of the root path of HDFs (Hdfs://hadoop01:9000/)

Import Java.io.FileInputStream;
Import Java.io.FileOutputStream;
Import Java.io.InputStream;
Import Java.io.OutputStream;
Import Java.net.URI;

Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.FileSystem;
Import Org.apache.hadoop.fs.Path;
Import Org.apache.hadoop.io.IOUtils;
Import Org.junit.Before;
Import Org.junit.Test;

public class Hadoopdemo {
private static FileSystem fs = null;
@Before
public void Init () throws exception{
If there is no third parameter, there will be no permission error when uploading the file, but it is not safe to use
Hadoop provides a framework called Kerberos to address its security concerns
FS = Filesystem.get (New URI ("hdfs://hadoop1:9000"), New Configuration (), "root");
}

/*
* Download File
*/
@Test
public void DownLoad () throws exception{
InputStream in = Fs.open (new Path ("/jdk.bin")); Open an input stream
OutputStream out = new FileOutputStream ("c:\\jdk_1.7"); Create a file output stream
Ioutils.copybytes (in, out, 4096, true); The third parameter represents the number of bytes cached, and the fourth parameter indicates that the stream is automatically closed after the current download is complete
}

/*
* Upload Files
*/
@Test
public void UpLoad () throws exception{
InputStream in = new FileInputStream ("E://android-sdk_r24.3.3-windows.zip"); Create a local file input stream
OutputStream out = fs.create (new Path ("/ni/android.zip")); To create an HDFS output stream
Ioutils.copybytes (in, out, 4096, true);
}

/*
* Delete Files
*/
@Test
public void Delete () throws exception{
Fs.delete (New Path ("/jdk.bin"), true);
}
public static void Main (string[] args) throws Exception {
TODO auto-generated Method Stub

}

}

Some basic operations for Hadoop