Hadoop executes HelloWorld to further execute file queries in HDFs

Preparatory work:

1, install the Hadoop;

2. Create a Helloworld.jar package, this article creates a jar package under the Linux shell:

Writing Helloworld.java files
public class HelloWorld
{
public static void Main (String []args) throws Exception
{
System.out.println ("Hello World");
}

}

Javac Helloworld.java is compiled and gets Helloworld.class
In the catalogue CV MANIFEST.MF file:
manifest-version:1.0
CREATED-BY:JDK1.6.0_45 (Sun Microsystems Inc.)
Main-class:helloworld

Run command: Jar CVFM Helloworld.jar MANIFEST. MF Helloworld.class

Run Java-jar Helloworld.jar to see the results of the run

Perform:

1. Hadoop executes Helloworld.jar file, CD to Helloworld.jar directory, executes Hadoop jar Helloworld.jar, execution result is same as Java-jar Helloworld.jar result.

According to the HelloWorld execution process, test the HDFS system file read operation, the code is as follows:

Import Java.io.InputStream;
Import Java.net.URI;

Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.FileSystem;
Import Org.apache.hadoop.fs.Path;
Import Org.apache.hadoop.io.IOUtils;
public class Filesystemcat {public

  static void Main (string[] args) throws Exception {
    String uri = args[0];
    Configuration conf = new configuration ();
    FileSystem fs = Filesystem.get (Uri.create (URI), conf);
    InputStream in = null;
    try {in
      = Fs.open (new Path (URI));
      Ioutils.copybytes (in, System.out, 4096, false);
    } finally {
      ioutils.closestream (in);}}
}

1. Before compiling this Filesystemcat.java file on Linux, you need to set the classpath:

1) Run the Hadoop classpath command to view the classpath required by Hadoop and get the following results on my computer:

/opt/hadoop-2.2.0/etc/hadoop:/opt/hadoop-2.2.0/share/hadoop/common/lib/*:/opt/hadoop-2.2.0/share/hadoop/common /*:/opt/hadoop-2.2.0/share/hadoop/hdfs:/opt/hadoop-2.2.0/share/hadoop/hdfs/lib/*:/opt/hadoop-2.2.0/share/ hadoop/hdfs/*:/opt/hadoop-2.2.0/share/hadoop/yarn/lib/*:/opt/hadoop-2.2.0/share/hadoop/yarn/*:/opt/ Hadoop-2.2.0/share/hadoop/mapreduce/lib/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/*:/opt/hadoop-2.2.0/contrib /capacity-scheduler/*.jar

2) write the above results to classpath, add the/etc/profile file

Export classpath=.: $JAVA _home/lib: $JAVA _home/jre/lib:/opt/hadoop-2.2.0/etc/hadoop:/opt/hadoop-2.2.0/share/ hadoop/common/lib/*:/opt/hadoop-2.2.0/share/hadoop/common/*:/opt/hadoop-2.2.0/share/hadoop/hdfs:/opt/ Hadoop-2.2.0/share/hadoop/hdfs/lib/*:/opt/hadoop-2.2.0/share/hadoop/hdfs/*:/opt/hadoop-2.2.0/share/hadoop/yarn /lib/*:/opt/hadoop-2.2.0/share/hadoop/yarn/*:/opt/hadoop-2.2.0/share/hadoop/mapreduce/lib/*:/opt/hadoop-2.2.0/ Share/hadoop/mapreduce/*:/opt/hadoop-2.2.0/contrib/capacity-scheduler/*.jar: $CLASSPATH

3) Execute Source/etc/profile under shell

2, Javac Filesystemcat.java compile Java files;

3, Jar CVFM Filesystemcat.jar MANIFEST. MF Filesystemcat.class Create Filesystemcat.jar package;

4. Create the input directory in the HDFS system and copy the Helloworld.java file into the HDFS system:

Hadoop Fs-mkdir Input

Hadoop fs-copyfromlocal./helloworld.java/input

5, Hadoop jar Filesystemcat.jar/input/helloworld.java can see the contents of the Helloworld.java file, note the/input/here Helloworld.java is in the HDFS system, the default full path is

Fs.defaultfs path +/input/helloworld.java in Core-site.xml, if configured as follows

<property>
       <name>fs.defaultFS</name>
       <value>hdfs://cloud001:9000</value>
</property>

We can also use the Hadoop jar Filesystemcat.jar Hdfs://cloud001:9000/input/helloworld.java to run.