Common Java APIs of HDFS

1. Use hadoop URL to read data

package hadoop;import java.io.InputStream;import java.net.URL;import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;import org.apache.hadoop.io.IOUtils;public class URLCat {    static {        URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());    }    public static void readHdfs(String url) throws Exception {        InputStream in = null;        try {            in = new URL(url).openStream();            IOUtils.copyBytes(in, System.out, 4096, false);        } finally {            IOUtils.closeStream(in);        }    }        public static void main(String[] args) throws Exception {        readHdfs("hdfs://192.168.49.131:9000/user/hadoopuser/input20120828/file01");    }}

The jar packages I use include:

The hadoop-core version must be consistent with the hadoop version installed in the distributed environment. Otherwise, an error is returned:

12/09/11 14:18:59 Info Security. usergroupinformation: JAAS configuration already set up for hadoop, not re-installing.
Exception in thread "Main" Java. Lang. noclassdeffounderror: ORG/Apache/hadoop/thirdparty/guava/common/collect/FIG
At org. Apache. hadoop. HDFS. socketcache. <init> (socketcache. Java: 48)
At org. Apache. hadoop. HDFS. dfsclient. <init> (dfsclient. Java: 240)

The hadoop version installed in the distributed environment is as follows: