Hadoop Maximum integer algorithm

Environment:

Linux system centOS6.3 (64bit)

hadoop1.1.2

Version of Eclipse under Linux

Maximum Algorithm code:

Package yunsave;


Import java.io.IOException;
Import org.apache.hadoop.conf.Configuration;
Import org.apache.hadoop.conf.Configured;
Import Org.apache.hadoop.fs.Path;
Import org.apache.hadoop.io.LongWritable;
Import org.apache.hadoop.io.IntWritable;
Import Org.apache.hadoop.io.Text;
Import Org.apache.hadoop.mapreduce.Job;
Import Org.apache.hadoop.mapreduce.Mapper;
Import Org.apache.hadoop.mapreduce.Reducer;
Import Org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
Import Org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
Import Org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
Import Org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
Import Org.apache.hadoop.util.Tool;
Import Org.apache.hadoop.util.ToolRunner;
To find the maximum value
public class MaxValue extends configured implements Tool {
public static class Mapclass extends Mapper<longwritable, Text, intwritable, intwritable> {
private int maxnum = 0;
public void Map (longwritable key, Text value, context context)
Throws IOException, Interruptedexception {
string[] str = value.tostring (). Split ("");
try {//for non-numeric characters we ignore
for (int i=0;i<str.length;i++) {
int temp = Integer.parseint (Str[i]);
if (Temp > Maxnum) {
Maxnum = temp;
}
}
} catch (NumberFormatException e) {
}
}
@Override
protected void Cleanup (context context) throws IOException,
interruptedexception {
Context.write (New Intwritable (Maxnum), New Intwritable (Maxnum));
}
}
public static class Reduce extends Reducer<intwritable, intwritable, intwritable, intwritable> {
private int maxnum = 0;
public void reduce (intwritable key, iterable<intwritable> values, context context)
Throws IOException, Interruptedexception {
for (intwritable val:values) {
if (Val.get () > Maxnum) {
Maxnum = Val.get ();
}
}
}
@Override
protected void Cleanup (context context) throws IOException,
interruptedexception {
Context.write (New Intwritable (Maxnum), New Intwritable (Maxnum));
}
}
public int run (string[] args) throws Exception {
Configuration conf = getconf ();
Job Job = new Job (conf, "Maxnum");
Job.setjarbyclass (Maxvalue.class);
Fileinputformat.setinputpaths (Job, New Path (Args[0]));
Fileoutputformat.setoutputpath (Job, New Path (Args[1]));
Job.setmapperclass (Mapclass.class);
Job.setcombinerclass (Reduce.class);
Job.setreducerclass (Reduce.class);
Job.setinputformatclass (Textinputformat.class);
Job.setoutputformatclass (Textoutputformat.class);
Job.setoutputkeyclass (Intwritable.class);
Job.setoutputvalueclass (Intwritable.class);
System.exit (Job.waitforcompletion (true)? 0:1);
return 0;
}
public static void Main (string[] args) throws Exception {
Long start = System.nanotime ();
int res = Toolrunner.run (new Configuration (), New MaxValue (), args);
System.out.println (System.nanotime ()-start);
System.exit (RES);
}
}

The file content entered:

[Email protected] ~]$ Hadoop fs-cat input_20141107/555.txt
Warning: $HADOOP _home is deprecated.


1 5 10 9999
[Email protected] ~]$

[Email protected] ~]$ Hadoop fs-cat input_20141107/666.txt
Warning: $HADOOP _home is deprecated.


111 222 333 888
[Email protected] ~]$

Eclipse's execution screen:

1.Argument parameters

Program Arguments:

hdfs://master:9000/user/work/input_20141107 hdfs://master:9000/user/work/output_20141107

VM Arguments:

-xms512m-xmx1024m-xx:maxpermsize=256m

Click Run to produce the following results

Operation Result:

[Email protected] ~]$ Hadoop fs-cat output_20141107/part-r-00000
Warning: $HADOOP _home is deprecated.


9999 9999
[Email protected] ~]$

Hadoop Maximum integer algorithm