Write MapReduce jobs in Python

Using Python to write MapReduce job mrjob allows you to use Python 2.5 + to write MapReduce jobs and run them on multiple different platforms. you can:

Write multi-step MapReduce jobs using pure Python

Test on local machine

Run on Hadoop cluster

Use Amazon Elastic MapReduce (EMR) to run on the cloud

The installation method of pip is very simple. you do not need to configure it. run pip install mrjob directly.

Code example:

From mrjob. job import MRJobclass MRWordCounter (MRJob): def mapper (self, key, line): for word in line. split (): yield word, 1 def reducer (self, word, occurrences): yield word, sum (occurrences) if _ name _ = '_ main _': MRWordCounter. run ()