eclipse 運行 mapreduce程式報錯 No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).

標籤：包名   tool   打包   org   leo   mapr   cli   lips   mapreduce   

報錯資訊

17/07/06 17:00:27 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
17/07/06 17:00:27 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
17/07/06 17:00:27 INFO input.FileInputFormat: Total input paths to process : 1
17/07/06 17:00:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/07/06 17:00:27 WARN snappy.LoadSnappy: Snappy native library not loaded
17/07/06 17:00:27 INFO mapred.JobClient: Running job: job_201707060106_0012
17/07/06 17:00:28 INFO mapred.JobClient: map 0% reduce 0%
17/07/06 17:00:34 INFO mapred.JobClient: Task Id : attempt_201707060106_0012_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mapreduce.MapUtil

 

問題出在工程下沒有job 的jar包

解決辦法

在conf中添加 conf.set("mapred.jar", "hadooptest.jar"); 其中 hadooptest.jar為匯出jar包名稱，mapred.jar不變

將工程打包jar檔案，放到工程根目錄下，再次運行問題解決

源碼

 

 

package com.mapreduce;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class MapReduceMain {

public static void main(String[] args) {
Configuration conf = new Configuration();
conf.set("mapred.job.tracker", "test1:9001");
conf.set("mapred.jar", "hadooptest.jar");
try {
Job job = new Job(conf);
job.setJarByClass(MapReduceMain.class);//設定啟動類
job.setMapperClass(MapUtil.class);//設定map類
job.setReducerClass(ReduceUtil.class);//設定reduce類
job.setOutputKeyClass(Text.class);//設定輸出key類型類
job.setOutputValueClass(IntWritable.class);//設定資料value類型類
//job.setNumReduceTasks(1);//設定reduce任務個數，預設為1
//輸入資料所在的檔案目錄
FileInputFormat.addInputPath(job, new Path("hdfs://test1:9000/input/"));
//mapreduce執行後輸出資料目錄
FileOutputFormat.setOutputPath(job, new Path("hdfs://test1:9000/output/"));
System.exit(job.waitForCompletion(true)?0:1);
} catch (Exception e) {
e.printStackTrace();
}
}

}

eclipse 運行 mapreduce程式報錯 No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).