Hadop after using partitioner, the result is still a file, how to solve??

Source: Internet
Author: User

Recently looked at the partitioner, so according to write a case, finally found that the program did not write the results separately to the corresponding file, the result is a file, so it is not a cluster to run the program, found control or local code execution:

<span style= "FONT-SIZE:12PX;" >2015-08-09 09:53:02,193 WARN [main] Conf. Configuration (Configuration.java:loadProperty (2172))-file:/tmp/hadoop-cau/mapred/staging/cau1745029252/. Staging/job_local1745029252_0001/job.xml:an attempt to override final parameter:  Mapreduce.job.end-notification.max.retry.interval; ignoring.2015-08-09 09:53:02,195 WARN [main] Conf. Configuration (Configuration.java:loadProperty (2172))-file:/tmp/hadoop-cau/mapred/staging/cau1745029252/. Staging/job_local1745029252_0001/job.xml:an attempt to override final parameter:  mapreduce.job.end-notification.max.attempts; ignoring.2015-08-09 09:53:02,632 WARN [main] Conf. Configuration (Configuration.java:loadProperty (2172))-File:/tmp/hadoop-cau/mapred/local/localrunner/cau/job_ Local1745029252_0001/job_local1745029252_0001.xml:an attempt to override final parameter:  Mapreduce.job.end-notification.max.retry.interval; ignoring.2015-08-09 09:53:02,634 WARN [main] Conf. Configuration (Configuration.java:loadPropeRty (2172))-file:/tmp/hadoop-cau/mapred/local/localrunner/cau/job_local1745029252_0001/job_local1745029252_  0001.xml:an attempt to override final parameter:mapreduce.job.end-notification.max.attempts; ignoring.2015-08-09 09:53:02,657 INFO [main] MapReduce. Job (Job.java:submit (1272))-the URL to track the job:http://localhost:8080/2015-08-09 09:53:02,659 INFO [main] mapredu Ce. Job (Job.java:monitorAndPrintJob (1317))-Running job:job_local1745029252_00012015-08-09 09:53:02,662 INFO [Thread-12 ] Mapred. Localjobrunner (LocalJobRunner.java:createOutputCommitter (323))-outputcommitter set in config null2015-08-09 09:53:02,679 INFO [Thread-12] mapred. Localjobrunner (LocalJobRunner.java:createOutputCommitter (341))-Outputcommitter is org.apache.hadoop.mapreduce.lib.output.fileoutputcommitter2015-08-09 09:53:02,810 INFO [Thread-12] mapred. Localjobrunner (LocalJobRunner.java:run (389))-Waiting for map tasks</span>


As a result, think of packaging to the cluster to run to see, the results of the node reported a variety of errors! :


<span style= "FONT-SIZE:12PX;" >15/08/09 10:02:41 INFO MapReduce. Job:the URL to track the job:http://master:8088/proxy/application_1438842873940_0002/15/08/09 10:02:41 INFO mapreduce. Job:running job:job_1438842873940_000215/08/09 10:02:54 INFO mapreduce. Job:job job_1438842873940_0002 running in Uber mode:false15/08/09 10:02:54 INFO MapReduce. Job:map 0% reduce 0%15/08/09 10:02:55 INFO mapreduce. Job:task Id:attempt_1438842873940_0002_m_000000_0, Status:failedcontainer launch failed for container_1438842873940_ 0002_01_000002:java.lang.illegalargumentexception:java.net.unknownhostexception:slave7at Org.apache.hadoop.security.SecurityUtil.buildTokenService (securityutil.java:377) at Org.apache.hadoop.security.SecurityUtil.setTokenService (securityutil.java:356) at Org.apache.hadoop.yarn.util.ConverterUtils.convertFromYarn (converterutils.java:237) at org.apache.hadoop.yarn.client.api.impl.containermanagementprotocolproxy$ Containermanagementprotocolproxydata.newproxy (containermanagementprotocolproxy.java:218) at org.apache.hadoop.yarn.client.api.impl.containermanagementprotocolproxy$ Containermanagementprotocolproxydata.<init> (containermanagementprotocolproxy.java:196) at Org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy.getProxy ( containermanagementprotocolproxy.java:117) at Org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl.getCMProxy (containerlauncherimpl.java:403) at Org.apache.hadoop.mapreduce.v2.app.launcher.containerlauncherimpl$container.launch (ContainerLauncherImpl.java : 138) at Org.apache.hadoop.mapreduce.v2.app.launcher.containerlauncherimpl$eventprocessor.run ( containerlauncherimpl.java:369) at Java.util.concurrent.ThreadPoolExecutor.runWorker (Threadpoolexecutor.java : 1145) at Java.util.concurrent.threadpoolexecutor$worker.run (threadpoolexecutor.java:615) at Java.lang.Thread.run ( thread.java:722) caused by:java.net.unknownhostexception:slave7</span>

Finally, the problem is still unresolved, which hero pointed it. Little brother's not getting started! In this heavens!!!

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Hadop after using partitioner, the result is still a file, how to solve??

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.