C + + programs under Hadoop-weather examples

Source: Internet
Author: User
Tags emit hdfs dfs

Very much want to be able to do C + + program on Hadoop. Oneself to C + + still a bit of plot, according to "The Hadoop authoritative guide Chinese second Edition" The Pipes of Hadoop has carried on the experiment, and tested successfully

#include <algorithm> #include <limits.h> #include <stdint.h> #include <string> #include Pip Es.hh "#include" templatefactory.hh "#include" stringutils.hh "class Maxtemperaturemapper:public Hadooppipes::mappe R {Public:maxtemperaturemapper (hadooppipes::taskcontext& context) {} void Map (Hadooppipes::mapcontext&amp ;      Context) {std::string line = Context.getinputvalue ();      Std::string year = LINE.SUBSTR (15, 4);      std::string airtemperature = Line.substr (87, 5);      std::string q = LINE.SUBSTR (92, 1); if (airtemperature! = "+9999" && (q = = "0" | | q = = "1" | | q = "4" | | q = "5" | | q = "9") {con      Text.emit (year, airtemperature);    }    }  }; Class Maptemperaturereducer:public Hadooppipes::reducer {public:maptemperaturereducer (HadoopPipes::TaskContext&am P      context) {} void Reduce (hadooppipes::reducecontext& context) {int maxValue = int_min; while (Context.nextvalUE ()) {MaxValue = Std::max (MaxValue, Hadooputils::toint (Context.getinputvalue ()));    } context.emit (Context.getinputkey (), hadooputils::tostring (MaxValue));    }  };                                 int main (int argc, char *argv[]) {return Hadooppipes::runtask (Hadooppipes::templatefactory<maxtemperaturemapper,  Maptemperaturereducer> ());   }
Note: Not the same place as the book: limit.h header file

Makefile file (changed by yourself):

. Suffixes:.h. C. cpp. occ=g++cppflags =-m64 RM = Rmsrcs = Max_temperature.cppprogram = Max_temperatureinc_path =-I$ (HADOO P_dev_home)/includelib_path =-l$ (hadoop_dev_home)/lib/nativelibs =-lhadooppipes-lcrypto-lhadooputils-lpthread$ ( Program): $ (SRCS) $ (CC) $ (cppflags) $ (inc_path) $<-wall $ (lib_path) $ (LIBS)  -g-o2-o [email protected]. phony:cleanclean:$ (RM) $ (program)



Source data files:

0067011990999991950051507004+68750+023550fm-12+038299999v0203301n00671220001cn9999999n9+00001+99999999999
0043011990999991950051512004+68750+023550fm-12+038299999v0203201n00671220001cn9999999n9+00221+99999999999
0043011990999991950051518004+68750+023550fm-12+038299999v0203201n00261220001cn9999999n9-00111+99999999999
0043012650999991949032412004+62300+010750fm-12+048599999v0202701n00461220001cn0500001n9+01111+99999999999
0043012650999991949032418004+62300+010750fm-12+048599999v0202701n00461220001cn0500001n9+00781+99999999999

Uploaded to Hdfs:hdfs dfs-put sample.txt

Make becomes a running file uploaded to Hdfs:hdfs dfs-put max_temperature/bin


Operating method: Hadoop pipes-d hadoop.pipes.java.recordreader=true-d hadoop.pipes.java.recordwriter=true-input/user/root/ Sample.txt-output/output-program/bin/max_temperature
Data output results:



C + + programs under Hadoop-weather examples

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.