Java.io.IOException:Type mismatch in key from map: Workaround

Source: Internet
Author: User

An exception occurred while executing the MR Program:java.lang.Exception:java.io.IOException:Type mismatch in key from map:expected Org.apache.hadoop.io.Text, received org.apache.hadoop.io.LongWritable

The logs are as follows:

2016-05-09 21:33:28,871 INFO [org.apache.hadoop.conf.Configuration.deprecation]-session.id is deprecated. Instead, use Dfs.metrics.session-id 2016-05-09 21:33:28,873 INFO [Org.apache.hadoop.metrics.jvm.JvmMetrics]- Initializing JVM Metrics with Processname=jobtracker, sessionid= 2016-05-09 21:33:29,309 WARN [  Org.apache.hadoop.mapreduce.JobResourceUploader]-No job jar file set. User classes May is not found.  See Job or Job#setjar (String). 2016-05-09 21:33:29,494 INFO [Org.apache.hadoop.mapreduce.lib.input.FileInputFormat]-Total input paths to process:1 2  016-05-09 21:33:29,584 Info [Org.apache.hadoop.mapreduce.JobSubmitter]-Number of splits:1 2016-05-09 21:33:29,679 info [Org.apache.hadoop.mapreduce.JobSubmitter]-Submitting tokens for job:job_local1411634813_0001 2016-05-09 21:33:29,890 INFO [Org.apache.hadoop.mapreduce.Job]-the URL to track the job:http://localhost:8080/2016-05-09 21:33:2 9,891 INFO [Org.apache.hadoop.mapreduce.Job]-Running job:job_local1411634813_0001 2016-05-09 21:33:29,892 INFO [Org.apache.hadoop.mapred.LocalJobRunner]-outputcommitter set in config null 2 016-05-09 21:33:29,901 INFO [Org.apache.hadoop.mapred.LocalJobRunner]-Outputcommitter is Org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter 2016-05-09 21:33:30,000 INFO [ Org.apache.hadoop.mapred.LocalJobRunner]-Waiting for map tasks 2016-05-09 21:33:30,000 INFO [org.apache.hadoop.mapred . Localjobrunner]-Starting task:attempt_local1411634813_0001_m_000000_0 2016-05-09 21:33:30,035 INFO [  Org.apache.hadoop.yarn.util.ProcfsBasedProcessTree]-Procfsbasedprocesstree currently is supported only on Linux. 2016-05-09 21:33:30,081 INFO [Org.apache.hadoop.mapred.Task]-Using resourcecalculatorprocesstree: [email  Protected] 2016-05-09 21:33:30,088 INFO [org.apache.hadoop.mapred.MapTask]-processing split:hdfs:// 192.168.5.97:8020/tmp/htb/mr/join_in/child-parent.txt:0+161 2016-05-09 21:33:30,144 INFO [ Org.apache.hadoop.mapred.MapTask]-(EQUATOR) 0KVI 26214396 (104857584) 2016-05-09 21:33:30,144 INFO [Org.apache.hadoop.mapred.MapTask]-MAPREDUCE.TASK.IO.SORT.MB: 2016-05-09 21:33:30,144 INFO [org.apache.hadoop.mapred.MapTask]-Soft limit at 83886080 2016-05-09 21:33:30,144 INF O [Org.apache.hadoop.mapred.MapTask]-bufstart = 0; bufvoid = 104857600 2016-05-09 21:33:30,145 INFO [Org.apache.hadoop.mapred.MapTask]-kvstart = 26214396; Length = 6553600 2016-05-09 21:33:30,148 INFO [org.apache.hadoop.mapred.MapTask]-Map output collector class = Org.apach E.hadoop.mapred.maptask$mapoutputbuffer 2016-05-09 21:33:30,462 INFO [Org.apache.hadoop.mapred.MapTask]-Starting  Flush of map output 2016-05-09 21:33:30,479 INFO [Org.apache.hadoop.mapred.LocalJobRunner]-Map task executor complete. 2016-05-09 21:33:30,503 WARN [Org.apache.hadoop.mapred.LocalJobRunner]-job_local1411634813_0001 Java.lang.Exception:java.io.IOException:Type mismatch in key from map:expected Org.apache.hadoop.io.Text, received org . apache.hadoop.io.LoNgwritable</span>at Org.apache.hadoop.mapred.localjobrunner$job.runtasks (LocalJobRunner.java:462) at Org.apache.hadoop.mapred.localjobrunner$job.run (localjobrunner.java:522) caused By:java.io.IOException:Type Mismatch in key from Map:expected Org.apache.hadoop.io.Text, received org.apache.hadoop.io.longwritable</span> At Org.apache.hadoop.mapred.maptask$mapoutputbuffer.collect (maptask.java:1069) at Org.apache.hadoop.mapred.maptask$newoutputcollector.write (maptask.java:712) at Org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write (taskinputoutputcontextimpl.java:89) at Org.apache.hadoop.mapreduce.lib.map.wrappedmapper$context.write (wrappedmapper.java:112) at Org.apache.hadoop.mapreduce.Mapper.map (mapper.java:124) at Org.apache.hadoop.mapreduce.Mapper.run (Mapper.java : 145) at Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:784) at Org.apache.hadoop.mapred.MapTask.run ( maptask.java:341) at Org.apache.hadoop.mapred.localjobrunner$job$maptaskrunnable.run (LocalJobRunner.java:243) at Java.util.concurrent.executors$runnableadapter.call (Unknown Source) at Java.util.concurrent.FutureTask.run (Unknown Source) at Java.util.concurrent.ThreadPoolExecutor.runWorker (Unknown SOURCE) at Java.util.concurrent.threadpoolexecutor$worker.run (Unknown source) at Java.lang.Thread.run (Unknown source ) 2016-05-09 21:33:30,895 INFO [Org.apache.hadoop.mapreduce.Job]-Job job_local1411634813_0001 running in Uber mode:fals e 2016-05-09 21:33:30,896 Info [org.apache.hadoop.mapreduce.Job]-map 0% reduce 0% 2016-05-09 21:33:30,898 INFO [org.a PACHE.HADOOP.MAPREDUCE.JOB]-Job job_local1411634813_0001 failed with state failed due To:na 2016-05-09 in FO [Org.apache.hadoop.mapreduce.Job]-counters:0

The map function code is as follows

public static class Map extends Mapper {//implement map function public void map (Object key, Text value, Context context) throws Ioexcepti On, interruptedexception {string childname = new String (); String parentname = new string (); String relationtype = new string ();//input a line of preprocessed text StringTokenizer ITR = new StringTokenizer (value.tostring ()); String[] values = new String[2];int i = 0;while (Itr.hasmoretokens ()) {Values[i] = Itr.nexttoken (); i++;} if (Values[0].compareto ("child")! = 0) {childname = values[0];p arentname = values[1];//output left table Relationtype = "1"; context.wr Ite (new text (Values[1]), new text (Relationtype + "+" + childname + "+" + parentname),////output right Table Relationtype = "2"; context.w Rite (new text (Values[0]), new text (Relationtype + "+" + childname + "+" + ParentName));}}}

It is not possible to discover that the output type is not specified because the Mapper function is inherited here . because the default input format Textinputformat the resulting key type is longwritable, you should know how to change it here. Still do not know how to change, refer to the following



Java.io.IOException:Type mismatch in key from map: Workaround

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.