Java Client Development for Hadoop2.4.1 HDFs

I am developing this program in the Linux environment Eclipse, if you are writing this program in the Windows environment, please adjust it yourself.

First step: First we determine the environment of our Hadoop HDFs is good, we start HDFs in Linux, and then pass the URL test on the Web page: http://uatciti:50070

Step two: Open Eclipse under Linux and write our client code.

Description: We have JDK files under the Linux system's/usr/local/devtool/java file. We want to upload this file to the JDK folder under the root directory of the HDFS system.

<span style= "FONT-SIZE:14PX;" >package Com.npf.hadoop.hdfs;import Java.io.fileinputstream;import Java.io.ioexception;import Org.apache.commons.io.ioutils;import Org.apache.hadoop.conf.configuration;import Org.apache.hadoop.fs.fsdataoutputstream;import Org.apache.hadoop.fs.filesystem;import Org.apache.hadoop.fs.Path; public class Hdfsclient {public static void main (string[] args) throws Exception {Loadlocalfile2hdfs (        ); }/** * Load a local file to HDFs * @throws IOException */public static void Loadlo Calfile2hdfs () throws ioexception{//get A instance of FileSystem Configuration conf = new C                Onfiguration ();                Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");                FileSystem fs = Filesystem.get (conf); Get input stream of local file FileInputStream input = new FileInputStream ("/usr/local/devtool/java/jdk-7           u79-linux-i586.rpm ");     Get output stream of dest path PATH = new Path ("Hdfs://uatciti:9000/jdk");                Fsdataoutputstream output = fs.create (path);                Load a local file to HDFs ioutils.copy (input, output);        System.out.println ("completed"); }}</span></span>

To run this code, we check the file under the HDFs system by visiting http://uatciti:50070 to determine if we have successfully uploaded it.

Ok, we have uploaded a file from the local system to the HDFS system.

Now we want to download the JDK from the root directory in the HDFs system to the/USR/LOCAL/DEVTOO/JAVA/JDK directory under the Linux Local system.

The code is as follows:

<span style= "FONT-SIZE:14PX;" >/** * Download file from HDFs to local * @throws IOException */public static V                        OID downloadfilefromhdfs2local () throws ioexception{//get a instance of FileSystem                        Configuration conf = new configuration ();                        Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");                        FileSystem fs = Filesystem.get (conf); Get input stream of HDFs file Fsdatainputstream input = Fs.open ("New Path" ("Hdfs://uatciti:9000/jdk                        ")); Get output stream of the local file FileOutputStream output = new FileOutputStream ("/usr/local/devtoo                        L/java/jdk ");                        Download file from HDFs to local ioutils.copy (input, output);        System.out.println ("completed"); }}</span> 

after running this code, check to see if the download was successful:

OK, we see that this file has been downloaded to the Linux Local system disk. We're done here.

We create a directory on the HDFS system by code:/aa/bb

<span style= "FONT-SIZE:14PX;" >       /**         * mkdir         * @throws ioexception         *        /public static void MkDir () throws ioexception{             // Get a instance of FileSystem             Configuration conf = new configuration ();             Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");             FileSystem fs = Filesystem.get (conf);             Fs.mkdirs (New Path ("/aa/bb"));        } </span>

After running this code, we can view it through http://uatciti:50070.

Our previous download method is somewhat complex, in fact HDFs has a good API for us to use, see the code below. We want to upload the JDK below/usr/local/devtool/java to the/aa/bb directory in HDFs.

<span style= "FONT-SIZE:14PX;" >public static void Downloadfile2hdfs () throws ioexception{      //get A instance of FileSystem      Configuration conf = new Configuration ();      Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");      FileSystem fs = Filesystem.get (conf);      Fs.copyfromlocalfile (New Path ("/usr/local/devtool/java/jdk-7u79-linux-i586.rpm"), New Path ("/aa/bb")); </span></span>

after running this code, we can view it through http://uatciti:50070.

Now we want to remove all the files under the/aa/bb directory from the existing BB folder, OK, see the code below.

<span style= "FONT-SIZE:14PX;" >/**   * Delete file   * @throws ioexception   */public static void DeleteFile () throws ioexception{        //get A instance of FileSystem        Configuration conf = new configuration ();        Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");        FileSystem fs = Filesystem.get (conf);        Cascade Delete all files under the/aa/bb/directory        fs.delete (new Path ("/aa/bb"), true);} </span></span>

Now we see that there is a file under the root directory called the JDK, and now our requirement is to rename the JDK file to JDKNPF. OK, look at the code below.

<span style= "FONT-SIZE:14PX;" >public static void Renamefilename () throws ioexception{       //get A instance of FileSystem       Configuration conf = NE W Configuration ();       Conf.set ("Fs.defaultfs", "hdfs://uatciti:9000");       FileSystem fs = Filesystem.get (conf);       Fs.rename (New Path ("/jdk"), New Path ("/JDKNPF")); </span>

After running this code, we can view it through http://uatciti:50070.

Java Client Development for Hadoop2.4.1 HDFs