Run the wordcount program of the hadoop instance through the command line.

Document directory

• 1. If the wordcount program does not contain layers, there is no package
• 2. If the wordcount program contains layers
• 3. Compile the wordcount. Java program
• 4. The wordcount. Java program cannot be compiled.

Reference 1: http://www.cnblogs.com/flying5/archive/2011/05/04/2078408.html

Note the following points.

1. If the wordcount program does not contain layers, there is no package

Use the following command:

hadoop jar wordcount.jar WordCount2 /home/hadoop/input/20418.txt /home/hadoop/output/wordcount2-6

The command line indicates that the hadoop program is executedWordcount. Jar. The wordcount. jar file contains the following class files:

• Wordcount. Class
• Wordcount $ map. Class
• Wordcount $ reduce. Class

The following four claas files are compiled with wordcount2.java:

• Classwordcount2.class
• Wordcount2 $ intsumreducer. Class
• Wordcount2 $ intwritabledecreasingcomparator. Class
• Wordcount2 $ tokenizermapper. Class

And these. class files are under the root directory of the jar package. To package the preceding seven class files into jar files, run the following command to import the seven class files in the wordcount folder of the same directory, and then run the following command to package them:

jar cvf WordCount.jar *.class

2. If the wordcount program contains layers

Use the following command (error ):

$ hadoop jar wordcount.jar WordCount2 /home/hadoop/input/20418.txt /home/hadoop/output/wordcount2-7

Will report an error, the error is as follows :( for specific reasons can refer to blog: http://blog.csdn.net/xw13106209/article/details/6861855)

Exception in thread "main" java.lang.ClassNotFoundException: WordCount2at java.net.URLClassLoader$1.run(URLClassLoader.java:202)at java.security.AccessController.doPrivileged(Native Method)at java.net.URLClassLoader.findClass(URLClassLoader.java:190)at java.lang.ClassLoader.loadClass(ClassLoader.java:306)at java.lang.ClassLoader.loadClass(ClassLoader.java:247)at java.lang.Class.forName0(Native Method)at java.lang.Class.forName(Class.java:247)at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

The correct command is as follows:

$ hadoop jar WordCount.jar org.apache.hadoop.examples.WordCount2 /home/hadoop/input/20418.txt /home/hadoop/output/wordcount2-7

The only jar packages of different colors are different. The jar package of the previous command is wordcount. jar, and the Java package here is wordcount. jar. Wordcount. Jar packages the entire directory of org/Apache/hadoop/examples.

3. Compile the wordcount. Java program

Use commands similar to the following in reference 1 to compile wordcount. Java

javac -classpath /home/hadoop/program/hadoop-0.20.1/hadoop-0.20.1-core.jar WordCount.java -d /home/hadoop/WordCount/

The classspath specified for this command is/home/hadoop/Program/hadoop-0.20.1/hadoop-0.20.1-core.jar. In fact, we can modify environment variables to save this content.

The specific method is:

 sudo gedit /etc/profile

Modify

export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH

Is

export HADOOP_HOME=/home/hadoop/program/hadoop-0.20.1export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH:$HADOOP_HOME/hadoop-0.20.1-core.jar

In this way, you can directly use the following command to compile the wordcount. Java program.

javac  WordCount.java -d /home/hadoop/WordCount/

Note:Wordcount. Java can have a package name or no package name. If there is no package name, there will be several compiled. class files in the/home/hadoop/wordcount/directory. If there is a package name, the structure directory of the package will be generated under the/home/hadoop/wordcount/directory.

4. The wordcount. Java program cannot be compiled.

When using the same compilation command

javac WordCount2.java -d /home/hadoop/WordCount/

The following error is prompted:

Wordcount2.java: 93: unable to access Org. apache. commons. CLI. options does not find Org. apache. commons. CLI. options class file string [] otherargs = new genericoptionsparser (Conf, argS) ^ 1 error

I checked some information. This error is mainly because some classes are used in wordcount2.java, and these classes are not registered in the classpath path. Therefore, this error occurs. Because in eclipse, the jar package added to the library in our bulid path is not just a jar of the hadoop-0.20.1-core.jar, there are many other jar, if you want to open all these jar packages through the command line, it is very troublesome. Ant is recommended for compilation. The specific use of ant may be mentioned in a later blog. We can use eclipse for compilation. Refer to the blog: compile Java files in eclipse.