Writing MapReduce jobs using Python

Mrjob allows you to write a MapReduce job with Python 2.5+ and run it on several different platforms, you can:

Write a multi-step MapReduce job using pure Python

Test on this machine

Running on a Hadoop cluster

Run on the cloud with Amazon Elastic MapReduce (EMR)

Pip installation method is very simple, no configuration, direct operation: Pip install Mrjob

code example:

From mrjob.job import Mrjobclass mrwordcounter (mrjob):    def mapper (self, Key, line): For        word in Line.split (): 
 yield Word, 1    def reducer (self, Word, occurrences):        yield Word, sum (occurrences) if __name__ = = ' __main__ ':    Mrwordcounter.run ()

