Hadoop detailed (ii) Java access HDFs

All the source code on the GitHub, Https://github.com/lastsweetop/styhadoop

Read data using Hadoop URL read

A simpler way to read HDFS data is to open a stream through the Java.net.URL, but before you call it beforehand The Seturlstreamhandlerfactory method is set to Fsurlstreamhandlerfactory (this factory takes the parse HDFs protocol), which can only be invoked once, so it is written in a static block. The copybytes of the Ioutils class is then called to copy the HDFs data stream to the standard output stream System.out, the first two parameters are well understood, one input, one output, the third is the cache size, and the fourth specifies whether the stream is closed after the copy is completed. We want to set this to false, the standard output stream does not close, we want to manually close the input stream.

Package com.sweetop.styhadoop;  
      
Import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;  
Import Org.apache.hadoop.io.IOUtils;  
      
Import Java.io.InputStream;  
Import Java.net.URL;  
      
/** 
 * Created with IntelliJ idea. 
 * User:lastsweetop 
 * date:13-5-31 * time 
 : Morning 10:16 * To change this 
 template use File | Settings | File Templates. 
 *
/public class Urlcat {  
      
    static {  
        url.seturlstreamhandlerfactory (new Fsurlstreamhandlerfactory ());  
    } Public  
      
    static void Main (string[] args) throws Exception {  
        inputstream in = null;  
        try {in  
            = new URL (Args[0]). OpenStream ();  
            Ioutils.copybytes (in, System.out, 4096, false);  
        finally {  
            ioutils.closestream (in);}}}  

Reading data using the FileSystem API

The first is to instantiate the FileSystem object, passing the FileSystem class's Get method, where a Java.net.URL and a configuration configuration are passed in.

Then filesystem can open a stream through a path object, followed by the example above

Package com.sweetop.styhadoop;  
      
Import org.apache.hadoop.conf.Configuration;  
Import Org.apache.hadoop.fs.FileSystem;  
Import Org.apache.hadoop.fs.Path;  
Import Org.apache.hadoop.io.IOUtils;  
      
Import Java.io.InputStream;  
Import Java.net.URI;  
      
/** 
 * Created with IntelliJ idea. 
 * User:lastsweetop 
 * date:13-5-31 * time 
 : Morning 11:24 * To change this 
 template use File | Settings | File Templates. 
 */Public
class Filesystemcat {public  
    static void Main (string[] args) throws Exception {  
        String uri=args[ 0];  
        Configuration conf=new Configuration ();  
        FileSystem Fs=filesystem.get (Uri.create (URI), conf);  
        InputStream In=null;  
        try {  
            In=fs.open (new Path (URI));  
            Ioutils.copybytes (in, System.out, 4096, false);   finally {  
            ioutils.closestream (in);}}}