Eclipse Configuring MapReduce Environment error

Source: Internet
Author: User

Beginner MapReduce, want to configure the MapReduce environment on eclipse, a lot of tutorials on the web, but after the tutorial, it does not work properly.

Encountered the following error:

15/10/17 20:10:39 INFO JVM. Jvmmetrics:initializing JVM Metrics with Processname=jobtracker, sessionid=
15/10/17 20:10:39 WARN mapred. Jobclient:no job jar file set. User classes May is not found. See jobconf (Class) or Jobconf#setjar (String).
15/10/17 20:10:40 INFO input. Fileinputformat:total input paths to Process:2
15/10/17 20:10:40 INFO mapred. Jobclient:running job:job_local_0001
15/10/17 20:10:40 INFO input. Fileinputformat:total input paths to Process:2
15/10/17 20:10:41 INFO mapred. MAPTASK:IO.SORT.MB = 100
15/10/17 20:10:41 INFO mapred. Maptask:data buffer = 79691776/99614720
15/10/17 20:10:41 INFO mapred. Maptask:record buffer = 262144/327680
15/10/17 20:10:41 INFO mapred. maptask:starting Flush of map output
15/10/17 20:10:41 INFO mapred. Maptask:finished spill 0
15/10/17 20:10:41 INFO mapred. TaskRunner:Task:attempt_local_0001_m_000000_0 is done. and is in the process of commiting
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Taskrunner:task ' Attempt_local_0001_m_000000_0 ' done.
15/10/17 20:10:41 INFO mapred. MAPTASK:IO.SORT.MB = 100
15/10/17 20:10:41 INFO mapred. Maptask:data buffer = 79691776/99614720
15/10/17 20:10:41 INFO mapred. Maptask:record buffer = 262144/327680
15/10/17 20:10:41 INFO mapred. maptask:starting Flush of map output
15/10/17 20:10:41 INFO mapred. Maptask:finished spill 0
15/10/17 20:10:41 INFO mapred. TaskRunner:Task:attempt_local_0001_m_000001_0 is done. and is in the process of commiting
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Taskrunner:task ' attempt_local_0001_m_000001_0 ' done.
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Merger:merging 2 sorted Segments
15/10/17 20:10:41 INFO mapred. Merger:down to the last Merge-pass, with 2 segments left of total size:52 bytes
15/10/17 20:10:41 INFO mapred. Localjobrunner:
15/10/17 20:10:41 INFO mapred. Jobclient:map 100% Reduce 0%
15/10/17 20:10:42 INFO mapred. TaskRunner:Task:attempt_local_0001_r_000000_0 is done. and is in the process of commiting
15/10/17 20:10:42 INFO mapred. Localjobrunner:
15/10/17 20:10:42 INFO mapred. Taskrunner:task Attempt_local_0001_r_000000_0 is allowed to commit now
15/10/17 20:10:42 INFO output. fileoutputcommitter:saved output of Task ' attempt_local_0001_r_000000_0 ' to hdfs://master:9000/user/hadoop/out99
15/10/17 20:10:42 INFO mapred. Localjobrunner:reduce > Reduce
15/10/17 20:10:42 INFO mapred. Taskrunner:task ' Attempt_local_0001_r_000000_0 ' done.
15/10/17 20:10:42 INFO mapred. Jobclient:map 100% Reduce 100%
15/10/17 20:10:42 INFO mapred. Jobclient:job complete:job_local_0001
15/10/17 20:10:42 INFO mapred. Jobclient:counters:14
15/10/17 20:10:42 INFO mapred. Jobclient:filesystemcounters
15/10/17 20:10:42 INFO mapred. jobclient:file_bytes_read=50343
15/10/17 20:10:42 INFO mapred. jobclient:hdfs_bytes_read=59
15/10/17 20:10:42 INFO mapred. jobclient:file_bytes_written=102356
15/10/17 20:10:42 INFO mapred. Jobclient:hdfs_bytes_written=24
15/10/17 20:10:42 INFO mapred. Jobclient:map-reduce Framework
15/10/17 20:10:42 INFO mapred. Jobclient:reduce input groups=3
15/10/17 20:10:42 INFO mapred. Jobclient:combine Output records=4
15/10/17 20:10:42 INFO mapred. Jobclient:map input records=2
15/10/17 20:10:42 INFO mapred. Jobclient:reduce Shuffle bytes=0
15/10/17 20:10:42 INFO mapred. Jobclient:reduce Output records=3
15/10/17 20:10:42 INFO mapred. Jobclient:spilled records=8
15/10/17 20:10:42 INFO mapred. Jobclient:map Output bytes=40
15/10/17 20:10:42 INFO mapred. Jobclient:combine input records=4
15/10/17 20:10:42 INFO mapred. Jobclient:map Output records=4
15/10/17 20:10:42 INFO mapred. Jobclient:reduce input records=4

Run the program for the Wordcount.java source code that comes with Hadoop

1. Right-click on the Wordcount.java to export the jar file to the project's root directory.

2. Add the exported Wordcount.jar file and right-click to add to BuildPath.

3. Add in the source code

..................
Public Static voidMain (string[] args)throwsException {Configuration conf=NewConfiguration (); Conf.set ( " Mapred.job.tracker", "192.168.2.1:9001 "); String[] Otherargs=Newgenericoptionsparser (conf, args). Getremainingargs (); if(Otherargs.length! = 2) {System.err.println ("Usage:wordcount <in> <out>"); System.exit (2); } Job Job=NewJob (conf, "word count"); Job.setjarbyclass (WordCount.class); Job.setmapperclass (tokenizermapper.class); Job.setcombinerclass (intsumreducer.class); Job.setreducerclass (intsumreducer.class); Job.setoutputkeyclass (Text.class); Job.setoutputvalueclass (intwritable.class); Fileinputformat.addinputpath (Job,NewPath (otherargs[0])); Fileoutputformat.setoutputpath (Job,NewPath (otherargs[1])); System.exit (Job.waitforcompletion (true) ? 0:1); }
.....................

Eclipse Configuring MapReduce Environment error

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.