Implementation of WordCount Error java.lang.ClassNotFoundException solution under Eclipse

Execute wordcount error under Eclipse java.lang.ClassNotFoundException

17/08/29 07:52:54 INFO Configuration.deprecation:fs.default.name is deprecated. Instead, use Fs.defaultfs

17/08/29 07:52:54 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

17/08/29 07:52:55 INFO Client. Rmproxy:connecting to ResourceManager at/192.168.93.130:8032

17/08/29 07:52:56 WARN MapReduce.  Jobresourceuploader:no job jar file set. User classes May is not found. See Job or Job#setjar (String).

17/08/29 07:52:56 INFO input. Fileinputformat:total input paths to Process:2

17/08/29 07:52:56 INFO MapReduce. Jobsubmitter:number of Splits:2

17/08/29 07:52:56 INFO Configuration.deprecation:mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address

17/08/29 07:52:56 INFO Configuration.deprecation:fs.default.name is deprecated. Instead, use Fs.defaultfs

17/08/29 07:52:56 INFO MapReduce. Jobsubmitter:submitting Tokens for job:job_1503848790903_0013

17/08/29 07:52:56 INFO mapred. Yarnrunner:job jar is not present. Not adding any jar to the list of resources.

17/08/29 07:52:56 INFO Impl. yarnclientimpl:submitted Application application_1503848790903_0013

17/08/29 07:52:56 INFO MapReduce. Job:the URL to track the job:http://master:18088/proxy/application_1503848790903_0013/

17/08/29 07:52:56 INFO MapReduce. Job:running job:job_1503848790903_0013

17/08/29 07:53:04 INFO MapReduce. Job:job job_1503848790903_0013 running in Uber Mode:false

17/08/29 07:53:04 INFO MapReduce. Job:map 0% Reduce 0%

17/08/29 07:53:09 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000001_0, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

17/08/29 07:53:09 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000000_0, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

17/08/29 07:53:14 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000001_1, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

17/08/29 07:53:14 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000000_1, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

17/08/29 07:53:19 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000001_2, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

17/08/29 07:53:20 INFO MapReduce. Job:task Id:attempt_1503848790903_0013_m_000000_2, status:failed

Error:java.lang.RuntimeException:java.lang.ClassNotFoundException:Class com.asiainfo.demo.count.worldtest$ MyWorld not found

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2195)

At Org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass (jobcontextimpl.java:186)

At Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:745)

At Org.apache.hadoop.mapred.MapTask.run (maptask.java:341)

At Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:164)

At java.security.AccessController.doPrivileged (Native Method)

At Javax.security.auth.Subject.doAs (subject.java:422)

At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1698)

At Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:158)

caused By:java.lang.ClassNotFoundException:Class Com.asiainfo.demo.count.worldtest$myworld not found

At Org.apache.hadoop.conf.Configuration.getClassByName (configuration.java:2101)

At Org.apache.hadoop.conf.Configuration.getClass (configuration.java:2193)

... 8 more

Container killed by the Applicationmaster.

Container killed on request. Exit Code is 143

Container exited with a Non-zero exit code 143

17/08/29 07:53:25 INFO MapReduce. Job:map 100% Reduce 100%

17/08/29 07:53:25 INFO MapReduce. Job:job job_1503848790903_0013 failed with state failed due To:task failed task_1503848790903_0013_m_000001

Job failed as tasks failed. Failedmaps:1 failedreduces:0

17/08/29 07:53:25 INFO MapReduce. Job:counters:13

Job Counters

Failed Map Tasks=8

Killed reduce Tasks=1

Launched Map tasks=8

Other local map tasks=6

Data-local Map tasks=2

Total time spent by all maps in occupied slots (ms) =28393

Total time spent by all reduces in occupied slots (ms) =0

Total time spent by all map tasks (ms) =28393

Total time spent by all reduce tasks (ms) =0

Total Vcore-milliseconds taken by all map tasks=28393

Total Vcore-milliseconds taken by all reduce tasks=0

Total Megabyte-milliseconds taken by all map tasks=29074432

Total Megabyte-milliseconds taken by all reduce tasks=0

1

The solution is to turn your project into a jar package, and then set the Conf.set ("Mapred.jar", "/eclipse-jee/eclipse/demo.jar") on the configuration object;

or set the jar package path Job.setjar ("/eclipse-jee/eclipse/demo.jar") that the Job object executes;

Here's an example I wrote.

Configuration conf = new configuration ();

Conf.set ("Fs.default.name", "hdfs://192.168.93.130:9000");

Conf.set ("Mapred.job.tracker", "9001");

Conf.set ("Hadoop.job.user", "root");

Conf.set ("Mapreduce.framework.name", "yarn");

Conf.set ("Yarn.resourcemanager.address", "192.168.93.130:8032");

Conf.set ("Mapreduce.jobtracker.address", "192.168.93.130:9001");

Conf.set ("Yarn.resourcemanager.hostname", "192.168.93.130");

Conf.set ("Yarn.resourcemanager.admin.address", "192.168.93.130:8033");

Conf.set ("Yarn.resourcemanager.address", "192.168.93.130:8032");

Conf.set ("Yarn.resourcemanager.resource-tracker.address", "192.168.93.130:8036");

Conf.set ("Yarn.resourcemanager.scheduler.address", "192.168.93.130:8030");

Conf.set ("Mapred.jar", "/eclipse-jee/eclipse/demo.jar");

string[] Otherargs = new Genericoptionsparser (conf, args). Getremainingargs ();

if (otherargs.length! = 2) {

System.out.println ("Stage not exis input output");

System.exit (2);

}

Job Job = new Job (conf, "job" + calendar.getinstance (). Gettimeinmillis ());

Job.setjarbyclass (Worldtest.class);

Job.setjar ("/eclipse-jee/eclipse/demo.jar");

Job.setmapperclass (Myworld.class);

Job.setcombinerclass (Myreduce.class);

Job.setreducerclass (Myreduce.class);

Job.setoutputkeyclass (Text.class);

Job.setoutputvalueclass (Intwritable.class);

Fileinputformat.addinputpath (Job, New Path (Otherargs[0]));

Fileoutputformat.setoutputpath (Job, New Path (Otherargs[1]));

System.out.println (Job.waitforcompletion (true)? 0:1);

This article is from the "Generation Guru" blog, please make sure to keep this source http://765682.blog.51cto.com/755682/1960953

Implementation of WordCount Error java.lang.ClassNotFoundException solution under Eclipse