MapReduce Programming Example (ii) __ Programming

Prerequisite Preparation:

1.hadoop installation is operating normally. Hadoop installation Configuration Please refer to: Ubuntu under Hadoop 1.2.1 Configuration installation

2. The integrated development environment is normal. Integrated development environment Configuration Please refer to: Ubuntu building Hadoop Source Reading environment

MapReduce Programming Examples:

MapReduce Programming Example (i), detailing running the first MapReduce program in an integrated environment WordCount and Code Analysis

MapReduce Programming Example (ii), calculating average student scores

MapReduce Programming Example (iii), data deduplication

MapReduce Programming Example (iv), sorting

MapReduce Programming Example (v), MapReduce implements single-table association

MapReduce Programming Example (vi), MapReduce implements multi-table Association

Example Two, calculates the student's average performance, each file includes all student achievement, the format is the name result, has the number of subjects, how many input files.

As follows

Xiao Ming 23
Jack Bauer 57
Little Red 80
Little Fly 93
Little just 32
Little Wood 99

Implementation code:

Import java.io.IOException;
Import Java.util.Iterator;

Import Java.util.StringTokenizer;
Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.Path;
Import org.apache.hadoop.io.FloatWritable;
Import Org.apache.hadoop.io.Text;
Import Org.apache.hadoop.mapreduce.Job;
Import Org.apache.hadoop.mapreduce.Mapper;
Import Org.apache.hadoop.mapreduce.Reducer;
Import Org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
Import Org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

Import Org.apache.hadoop.util.GenericOptionsParser; /** * Calculates the student's average score * students ' grades are entered in a file per section * file contents are: Name score * @author DaT dev.tao@gmail.com * */public class Averagescore {PU Blic Static class Averagemapper extends Mapper<object, text, text, floatwritable>{@Override protected void M AP (Object key, Text value, Context context) throws IOException, interruptedexception {String line = value.tostring
			();
			StringTokenizer tokens = new StringTokenizer (line, "\ n"); while (Tokens.haSmoretokens ()) {String tmp = Tokens.nexttoken ();
				StringTokenizer sz = new StringTokenizer (TMP);
				String name = Sz.nexttoken ();
				FLOAT score = float.valueof (Sz.nexttoken ());
				Text outname = new text (name);//new, the set is always wrong, why is it not clear now?
				Floatwritable outscore = new floatwritable (score);
			Context.write (Outname, outscore); }}} public static class Averagereducer extends Reducer<text, floatwritable, Text, floatwritable>{@Over Ride protected void reduce (Text key, iterable<floatwritable> Value,context Context) throws IOException, Interr
			uptedexception {float sum = 0;
			int count = 0;
				for (floatwritable f:value) {sum + = F.get (); Count ++;//shuffle after must be < name,< score 1, score 2, score 3....>> So a value is certainly a discipline} floatwritable Averagescore = new Floatwri
			Table (sum/count);////new new, set is always wrong, specifically why now is not very clear.
		Context.write (key, Averagescore); }} public static void Main (string[] args) throws IOException, Classnotfoundexception, interruptedexception{System.out.println ("Begin");
		Configuration conf = new configuration ();
		string[] Otherargs = new Genericoptionsparser (conf, args). Getremainingargs ();
			if (otherargs.length<2) {System.out.println ("Please input at least 2 arguments");
		System.exit (2);
		Job Job = new Job (conf, "Average score");
		Job.setjarbyclass (Averagescore.class);
		Job.setmapperclass (Averagemapper.class);
		Job.setcombinerclass (Averagereducer.class);
		Job.setreducerclass (Averagereducer.class);
		Job.setoutputkeyclass (Text.class);
		
		Job.setoutputvalueclass (Floatwritable.class);
		Fileinputformat.addinputpath (Job, New Path (Otherargs[0]));
		
		Fileoutputformat.setoutputpath (Job, New Path (Otherargs[1]));
		
		System.exit (Job.waitforcompletion (true)? 0:1);
	System.out.println ("End");
 }
	
}

To configure the input and output parameters:

Hdfs://localhost:9000/user/dat/average_score_input Hdfs://localhost:9000/user/dat/average_score_output

Get the output result:

Little Just 65.333336
Jack Bauer 80.333336
Xiao Ming 48.333332
Little Wood 92.333336
Little Red 83.333336
Little Fly 83.0