MapReduce Programming Example (VI) __ Programming

Prerequisite Preparation:

1.hadoop installation is operating normally. Hadoop installation Configuration Please refer to: Ubuntu under Hadoop 1.2.1 Configuration installation

2. The integrated development environment is normal. Integrated development environment Configuration Please refer to: Ubuntu building Hadoop Source Reading environment

MapReduce Programming Examples:

MapReduce Programming Example (i), detailing running the first MapReduce program in an integrated environment WordCount and Code Analysis

MapReduce Programming Example (ii), calculating average student scores

MapReduce Programming Example (iii), data deduplication

MapReduce Programming Example (iv), sorting

MapReduce Programming Example (v), MapReduce implements single-table association
MapReduce Programming Example (vi), MapReduce implements multi-table Association


Multi-Table Association Description: Two table associations, as follows: Left table: Factoryname address
BMW Factory 2
Benz Factory 3
Voivo Factory 4
LG Factory 5

Right table: Addressid addressname
2 Beijing
3 Guangzhou
4 Shenzhen
5 Sanya

The Factoryname-address table is calculated based on the Addressid Association. Obviously, the left-right association is the same as a single-table association. Not much to express, there is a need to view the analysis of single-table association.

Package com.t.hadoop;
Import java.io.IOException;
Import java.util.ArrayList;

Import java.util.List;
Import org.apache.hadoop.conf.Configuration;
Import Org.apache.hadoop.fs.Path;
Import org.apache.hadoop.io.IntWritable;
Import Org.apache.hadoop.io.Text;
Import Org.apache.hadoop.mapreduce.Job;
Import Org.apache.hadoop.mapreduce.Mapper;
Import Org.apache.hadoop.mapreduce.Reducer;
Import Org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
Import Org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

Import Org.apache.hadoop.util.GenericOptionsParser;
	
	/** * Multi-table sort * @author DaT dev.tao@gmail.com * */public class Mtjoin {public static int times = 1;  public static class Mtmapper extends Mapper<object, text, text, text>{@Override protected void Map (Object key,
			Text value, Context context) throws IOException, interruptedexception {string relation = new String ();
			String line = value.tostring (); if (Line.contains ("Factoryname") | | Line.contains ("Addressid")) return;
			int i = 0; while (Line.charat (i) < ' 0 ' | |
			Line.charat (i) > ' 9 ') {i++;
				} if (i>0) {//left table relation = "1";
			Context.write (new text (String.valueof (Line.charat (i)), new text (Relation + line.substring (0,i-1)));
				}else{//Right Table relation = "2";
			Context.write (new text (String.valueof (Line.charat (i)), new text (Relation +line.substring (i+1)));  }}} public static class Mtreducer extends Reducer<text, text, text, text>{@Override protected void reduce (Text key, iterable<text> Value,context Context) throws IOException, interruptedexception {if (ti
				Mes==1) {context.write (new text ("Factoryname"), new text ("Address");
			Times + +;
			} int factorynum = 0;
			int addressnum = 0;
			string[] Factorys = new STRING[10];
			
			String[] addresses = new STRING[10];
					for (Text t:value) {if (T.charat (0) = = ' 1 ') {//Left table factorys[factorynum]=t.tostring (). substring (1);
				factorynum++; }else{//Right Table addresses[addrEssnum]=t.tostring (). substring (1);
				addressnum++; }} for (int i = 0;i<factorynum;i++) {for (int j=0;j<addressnum;j++) {context.write (new Text (factor
				Ys[i]), new Text (Addresses[j])); }}}} public static void Main (string[] args) throws IOException, ClassNotFoundException, Interruptedexce
		ption{Configuration conf = new configuration ();
		
		string[] Otherargs = new Genericoptionsparser (Conf,args). Getremainingargs ();
			if (otherargs.length<2) {System.out.println ("Parameters error");
		System.exit (2);
		} Job Job =new job (conf, "Mtjoin");
		Job.setjarbyclass (Mtjoin.class);
		Job.setmapperclass (Mtmapper.class);
		Job.setreducerclass (Mtreducer.class);
		Job.setoutputkeyclass (Text.class);
		
		Job.setoutputvalueclass (Text.class);
		Fileinputformat.addinputpath (Job, New Path (Otherargs[0]));
		
		Fileoutputformat.setoutputpath (Job, New Path (Otherargs[1]));
		
	System.exit (Job.waitforcompletion (true)? 0:1);
 }
}

Output Result: Factoryname Address
BMW Factory Beijing
Benz Factory Guangzhou
Voivo Factory Shenzhen
LG Factory Sanya

Welcome students to communicate more ~