"Who is the Big V" algorithm design (map-reduce TopN)

Job design and Data link: http://pan.baidu.com/s/1o6MJTyi Password: 628d

First, Job Requirements

Based on the watchlist Relsample.json file, the MapReduce algorithm was designed to get the top ten people who were most concerned, that is, looking for a big V.

Second, Algorithm Design

1. The first mapper is used to parse a JSON file:

Resolves the contents of the IDs list from the Relsample.json file, which is the list of people who are concerned about each person. The value of the output is each item in the list, and K is one.

The corresponding file is \src\mymapper\readjsonmapper.java.

1. Reducer counts the number of times each ID is followed based on 1 results:

The corresponding file is \src\myreducer\readjsonreducer.java.

1. The second mapper sorts the results of 2 and gets TOPN:

The corresponding file is \src\myreducer\ Topmapper.java.

Ps : Sorting Instructions (Description of the different parts of the JSONTOPN and jsonTopN3 two folders) There are two methods for sorting. The first method (corresponding to the JSONTOPN folder): Write a class to hold the result type (the result contains the ID and the number of two attributes to be followed), and then write a sorting algorithm to sort the class (based on the number of times of interest). Corresponding files: Jsontopn\src\mycommon \mydatacell.java Jsontopn\src\mycommon \ Mysort.java The second method (corresponding to the JsonTopN3 folder): Use Arrays.sort () to sort the classes that you define. The implementation is to first define its own class, which must inherit from the comparable extraction interface, because the object that calls sort sort must be comparable, defining its own comparison rules by overloading the CompareTo function of the comparable interface. Corresponding files: Jsontopn3\src\mycommon \mydatacell.java

Third, Run

1. JSONTOPN program operation

2. JsonTopN3 program operation

"Who is the Big V" algorithm design (map-reduce TopN)