Beginner MapReduce, want to configure the MapReduce environment on eclipse, a lot of tutorials on the web, but after the tutorial, it does not work properly.
Encountered the following error:
15/10/17 20:10:39 INFO JVM. Jvmmetrics:initializing JVM Metrics with Processname=jobtracker, sessionid=
15/10/17 20:10:39 WARN mapred. Jobclient:no job jar file set. User classes May is not found. See jobconf (Class) or Jobconf#setjar (String).
15/10/17 20:10:40 INFO input. Fileinputformat:total input paths to Process:2
15/10/17 20:10:40 INFO mapred. Jobclient:running job:job_local_0001
15/10/17 20:10:40 INFO input. Fileinputformat:total input paths to Process:2
15/10/17 20:10:41 INFO mapred. MAPTASK:IO.SORT.MB = 100
15/10/17 20:10:41 INFO mapred. Maptask:data buffer = 79691776/99614720
15/10/17 20:10:41 INFO mapred. Maptask:record buffer = 262144/327680
15/10/17 20:10:41 INFO mapred. maptask:starting Flush of map output
15/10/17 20:10:41 INFO mapred. Maptask:finished spill 0
15/10/17 20:10:41 INFO mapred. TaskRunner:Task:attempt_local_0001_m_000000_0 is done. and is in the process of commiting
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Taskrunner:task ' Attempt_local_0001_m_000000_0 ' done.
15/10/17 20:10:41 INFO mapred. MAPTASK:IO.SORT.MB = 100
15/10/17 20:10:41 INFO mapred. Maptask:data buffer = 79691776/99614720
15/10/17 20:10:41 INFO mapred. Maptask:record buffer = 262144/327680
15/10/17 20:10:41 INFO mapred. maptask:starting Flush of map output
15/10/17 20:10:41 INFO mapred. Maptask:finished spill 0
15/10/17 20:10:41 INFO mapred. TaskRunner:Task:attempt_local_0001_m_000001_0 is done. and is in the process of commiting
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Taskrunner:task ' attempt_local_0001_m_000001_0 ' done.
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Merger:merging 2 sorted Segments
15/10/17 20:10:41 INFO mapred. Merger:down to the last Merge-pass, with 2 segments left of total size:52 bytes
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Jobclient:map 100% Reduce 0%
15/10/17 20:10:42 INFO mapred. TaskRunner:Task:attempt_local_0001_r_000000_0 is done. and is in the process of commiting
15/10/17 20:10:42 INFO mapred. Localjobrunner:
15/10/17 20:10:42 INFO mapred. Taskrunner:task Attempt_local_0001_r_000000_0 is allowed to commit now
15/10/17 20:10:42 INFO output. fileoutputcommitter:saved output of Task ' attempt_local_0001_r_000000_0 ' to hdfs://master:9000/user/hadoop/out99
15/10/17 20:10:42 INFO mapred. Localjobrunner:reduce > Reduce
15/10/17 20:10:42 INFO mapred. Taskrunner:task ' Attempt_local_0001_r_000000_0 ' done.
15/10/17 20:10:42 INFO mapred. Jobclient:map 100% Reduce 100%
15/10/17 20:10:42 INFO mapred. Jobclient:job complete:job_local_0001
15/10/17 20:10:42 INFO mapred. Jobclient:counters:14
15/10/17 20:10:42 INFO mapred. Jobclient:filesystemcounters
15/10/17 20:10:42 INFO mapred. jobclient:file_bytes_read=50343
15/10/17 20:10:42 INFO mapred. jobclient:hdfs_bytes_read=59
15/10/17 20:10:42 INFO mapred. jobclient:file_bytes_written=102356
15/10/17 20:10:42 INFO mapred. Jobclient:hdfs_bytes_written=24
15/10/17 20:10:42 INFO mapred. Jobclient:map-reduce Framework
15/10/17 20:10:42 INFO mapred. Jobclient:reduce input groups=3
15/10/17 20:10:42 INFO mapred. Jobclient:combine Output records=4
15/10/17 20:10:42 INFO mapred. Jobclient:map input records=2
15/10/17 20:10:42 INFO mapred. Jobclient:reduce Shuffle bytes=0
15/10/17 20:10:42 INFO mapred. Jobclient:reduce Output records=3
15/10/17 20:10:42 INFO mapred. Jobclient:spilled records=8
15/10/17 20:10:42 INFO mapred. Jobclient:map Output bytes=40
15/10/17 20:10:42 INFO mapred. Jobclient:combine input records=4
15/10/17 20:10:42 INFO mapred. Jobclient:map Output records=4
15/10/17 20:10:42 INFO mapred. Jobclient:reduce input records=4
Run the program for the Wordcount.java source code that comes with Hadoop
1. Right-click on the Wordcount.java to export the jar file to the project's root directory.
2. Add the exported Wordcount.jar file and right-click to add to BuildPath.
3. Add in the source code
..................
Public Static voidMain (string[] args)throwsException {Configuration conf=NewConfiguration (); Conf.set ( " Mapred.job.tracker", "192.168.2.1:9001 "); String[] Otherargs=Newgenericoptionsparser (conf, args). Getremainingargs (); if(Otherargs.length! = 2) {System.err.println ("Usage:wordcount <in> <out>"); System.exit (2); } Job Job=NewJob (conf, "word count"); Job.setjarbyclass (WordCount.class); Job.setmapperclass (tokenizermapper.class); Job.setcombinerclass (intsumreducer.class); Job.setreducerclass (intsumreducer.class); Job.setoutputkeyclass (Text.class); Job.setoutputvalueclass (intwritable.class); Fileinputformat.addinputpath (Job,NewPath (otherargs[0])); Fileoutputformat.setoutputpath (Job,NewPath (otherargs[1])); System.exit (Job.waitforcompletion (true) ? 0:1); }
.....................
Eclipse Configuring MapReduce Environment error