Use C # To feel the charm of MongoDB mapreduce

With the increasing number of hadoop users, MapReduce is becoming more and more popular. MapReduce is a highlight of MongoDB. I also want to know more about MapReduce. In addition, MongoDB is easy to operate, so I chose it. MapReduce divides the problem into multiple different parts and distributes them to different servers for parallel processing. Each server returns the result to the master server after processing a part of the problem assigned to it, the master server summarizes the results and finally completes troubleshooting.

Map maps the document to a set and operates the document. This step may generate multiple keys and values or have nothing (the value to be processed in the document is blank ). Then, group by key and put the generated Value List into the corresponding key. Reduce, the values in the list are reduced to a value. This value is returned, and then the key group is continued to simplify until each key has only one value in the list. This value is the final result.

In MongoDB, you need to use JavaScript to write a Map () function and a Reduce () function. The Map function uses the emit function to return the value to be processed. Emit will pass a key and a value to the redcuce function. The Reduce function receives two parameters. The first parameter is the key, and the second parameter is the key returned by emit. The second parameter is the values array, that is, the value returned by emit, it consists of one or more documents corresponding to keys.

Let's write an instance and use the instance to understand MapReduce.

First, create an object class,

Next, connect to the database,

Insert several test records,

The following is the core of this example. Two JavaScript methods are mainly used to count the number of times different words appear in the set,

Use the following code to traverse the results returned by mapreduce:

Execution result

In this example, the mapreduce processing result method may be similar, and the execution process may be different, but the idea is such an idea,

During computing, mongodb generates several intermediate results and is automatically deleted after the calculation is completed,

You can write a JavaScript function to calculate the record in which the word appears,

The execution result is as follows (below the dotted line)