Error message: Call from 10.32.6.150:8020 failed on connection
If you are experiencing this problem, change your code in the normal case of the network as follows:
1System.setproperty ("Hadoop_user_name", "root"); 2Configuration conf =NewConfiguration (); 3Conf.set ("Fs.defaultfs", "hdfs://10.32.6.150:9000");4Conf.set ("Mapreduce.jobtracker.address", "hdfs://10.32.6.150:9001");5Conf.set ("Mapreduce.framework.name", "yarn");6Conf.set ("Mapreduce.app-submission.cross-platform", "true");//to solve cross-platform problems7Conf.set ("Fs.defaultfs", "hdfs://10.32.6.150:9000");8Conf.set ("Mapreduce.jobtracker.address", "hdfs://10.32.6.150:9001");9Conf.set ("Mapreduce.framework.name", "yarn");TenConf.set ("Mapreduce.app-submission.cross-platform", "true"); OneJob wordcountjob = job.getinstance (conf, "Mr"); A //Important: Specify the jar package where this job is located -Wordcountjob.setjar ("E:\\desktop\\wordcount1.jar"); -Wordcountjob.setjarbyclass (WordCount.class); the - //What class is the Mapper logical class used to set Wordcountjob -Wordcountjob.setmapperclass (Wordcountmapper.class); - //What class is the Reducer logical class used to set Wordcountjob +Wordcountjob.setreducerclass (Wordcountreducer.class); - + //set the KV data type of the map stage output AWordcountjob.setmapoutputkeyclass (Text.class); atWordcountjob.setmapoutputvalueclass (intwritable.class); - - //sets the KV data type of the final output -Wordcountjob.setoutputkeyclass (Text.class); -Wordcountjob.setoutputvalueclass (intwritable.class); - in //set the path where the text data to be processed is stored -Fileinputformat.setinputpaths (Wordcountjob, "Hdfs://10.32.6.150:9000/1.txt"); toFileoutputformat.setoutputpath (Wordcountjob,NewPath ("hdfs://10.32.6.150/outputsq111/")); + - //submit job to Hadoop cluster theWordcountjob.waitforcompletion (true);
1System.setproperty ("Hadoop_user_name", "root");2Configuration conf =NewConfiguration ();3Conf.set ("Fs.defaultfs", "hdfs://10.32.6.150:9000");4Conf.set ("Mapreduce.jobtracker.address", "hdfs://10.32.6.150:9001");5Conf.set ("Mapreduce.framework.name", "yarn");6Conf.set ("Mapreduce.app-submission.cross-platform", "true");7 //Create a Work object8Job Job = job.getinstance (conf, "WC");9 //Add a rack package pathTenJob.setjar ("E:\\desktop\\wordcount.jar"); OneJob.setjarbyclass (Wordcountother.class); A - //specifying the Map class and the reduce class -Job.setmapperclass (Wordcountmapper.class); theJob.setreducerclass (Wordcountreducer.class); - - //Specify the output type of the map -Job.setmapoutputkeyclass (Text.class); +Job.setmapoutputvalueclass (intwritable.class); - + //Specify the final output type AJob.setoutputkeyclass (Text.class); atJob.setoutputvalueclass (intwritable.class); - - //specify the input and output directory for the job -Fileinputformat.setinputpaths (Job,NewPath ("Hdfs://10.32.6.150:9000/1.txt")); -Fileoutputformat.setoutputpath (Job,NewPath ("hdfs://10.32.6.150:9000/output321/")); - in Booleanresult = Job.waitforcompletion (true); -System.exit (result?0:1);
Above is the error code, below is the code can be run, the two do not see what the difference, I repeatedly thought the network problem (because it is the intranet, think there will be a firewall, etc.), and then perform the next classmate write, run successfully
Solution: This problem can not find the cause, not the network problem, should not be the problem of eclipse, encountered on the re-write job code:)
Windows uses Eclipse cross-platform to run MapReduce prompt connection not on