Error message
17/07/06 17:00:27 WARN mapred. Jobclient:use Genericoptionsparser for parsing the arguments. Applications should implement Tool for the same.
17/07/06 17:00:27 WARN mapred. Jobclient:no job jar file set. User classes May is not found. See jobconf (Class) or Jobconf#setjar (String).
17/07/06 17:00:27 INFO input. Fileinputformat:total input paths to process:1
17/07/06 17:00:27 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable
17/07/06 17:00:27 WARN Snappy. Loadsnappy:snappy Native Library not loaded
17/07/06 17:00:27 INFO mapred. Jobclient:running job:job_201707060106_0012
17/07/06 17:00:28 INFO mapred. Jobclient:map 0% Reduce 0%
17/07/06 17:00:34 INFO mapred. Jobclient:task Id:attempt_201707060106_0012_m_000000_0, status:failed
Java.lang.RuntimeException:java.lang.ClassNotFoundException:com.mapreduce.MapUtil
The problem is that there is no job under the project Jar package
Solutions
Add Conf.set ("Mapred.jar", "Hadooptest.jar") in Conf; Where Hadooptest.jar is the export jar package name, Mapred.jar unchanged
Package the project jar file, put it in the project root directory, run the problem again
Source
Package com.mapreduce;
Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.Path;
Import org.apache.hadoop.io.IntWritable;
Import Org.apache.hadoop.io.Text;
Import Org.apache.hadoop.mapreduce.Job;
Import Org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
Import Org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
public class Mapreducemain {
public static void Main (string[] args) {
Configuration conf = new configuration ();
Conf.set ("Mapred.job.tracker", "test1:9001");
Conf.set ("Mapred.jar", "Hadooptest.jar");
try {
Job Job = new Job (conf);
Job.setjarbyclass (Mapreducemain.class);//Set Startup class
Job.setmapperclass (Maputil.class);//Set Map class
Job.setreducerclass (Reduceutil.class);//Set reduce class
Job.setoutputkeyclass (Text.class);//Set Output Key type class
Job.setoutputvalueclass (Intwritable.class);//Set data value type class
Job.setnumreducetasks (1);//Set the number of reduce tasks by default to 1
The file directory where the input data resides
Fileinputformat.addinputpath (Job, New Path ("hdfs://test1:9000/input/"));
MapReduce output Data Directory after execution
Fileoutputformat.setoutputpath (Job, New Path ("hdfs://test1:9000/output/"));
System.exit (Job.waitforcompletion (true)? 0:1);
} catch (Exception e) {
E.printstacktrace ();
}
}
}
Eclipse runs the MapReduce program with the error No job jar file set. User classes May is not found. See jobconf (Class) or Jobconf#setjar (String).