MongoDB MapReduce Usage Summary

article from my personal blog: MongoDB mapreduce use summary ?

As we all know, MongoDB is a non-relational database, that is, each table in the MongoDB database is independent, there is no dependency between the table and the table. In MongoDB, in addition to the various CRUD statements, we also provide aggregation and mapreduce statistics, this article mainly to talk about MongoDB's mapreduce operation.

I won't go into the concept of mapreduce, let's check it out for ourselves.

In MongoDB, the syntax for MapReduce is as follows:

?

Db.table.mapReduce (map, reduce, {query:query, out:out,///Specify how the result set is stored, optional                        Parameters include://replace: If the document (table) exists, replace table,//merge: Overwrites the existing document record if there is a record in the document Reduce: If a record of the same key exists in the document, then two records are calculated, and then the old record is overwritten//{Inline:1} stores the record in memory and does not write to disk (the amount of user data is less Sort:sort, Limit:limit, finalize:function//This function is primarily used to modify data before being stored out, function (            key,values) {//return modifiedvalues;}                                Scope:document,//Specify the range of documents that reduce can access Jsmode:boolean//Specify whether to convert data to Bason format immediately between map and Ruduce, false by default If you want to set to true, remember the official my caution://you can only use Jsmode for result SE            TS with fewer than//500,000 distinct key arguments to the mapper ' s emit () function.    Verbose:boolean//Whether to include timing information in the result set, which is included by default}) 

When making MongoDB MapReduce, make sure that your query is available to the index, otherwise, in the statistics of the big data, will be the whole database, if there is no way to build index, then in the result set to judge some non-conforming data, and remove the query.

The syntax of MapReduce is actually very simple, but there are a few things to note:

? ? 1. In map, MongoDB is every 1000 data to reduce once

? ? 2. In map, if you want to count the sum of a data, you need to write this:

?    ? ? Emit (Key:this.key,sum:0})

?    ? Then in reduce you need to add up the last sum iteration and return {sum:sum}; If you don't, the data you calculate is always counted after less than 1000 data, and the previous data is lost.

? 3. If you can not use MapReduce, it is not necessary, the program can be counted, not using MongoDB frequent statistics.

? The data format of the 4.mapreduce result set is: {_id:key,value:{}}, so if you want to use the table directly, it is best to re-organize the data format once, and try to put the data in the last, rather than using value.xxx to query.

? Here is a mapreduce that counts the number of users posted on our site for a reference value in one code format:

?

var db = connect (' 127.0.0.1:27017/test ');d b.aaccounttemp.drop (); var map = function () {emit (This.accountid, {su m:0, Reblogflag:this.reblogflag,dashboardflag:this.dashboardflag,dashboardtype:this.dashboardtype, p hotonum:0,postnum:0,reblognum:0,videonum:0,videoshortnum:0, musicnum:0, questionnum:0,appnum:0, dialogNum:0});}    var reduce = function (key,values) {var sum = 0;    var photonum = 0;    var postnum = 0;    var reblognum = 0;    var videonum = 0;    var videoshortnum = 0;    var musicnum = 0;    var questionnum = 0;    var appnum = 0;    var dialognum = 0;        for (var i = 0; i < values.length; i++) {var data = Values[i];        var reblogflag = Data.reblogflag;        var dashboardflag = Data.dashboardflag;        var dashboardtype = Data.dashboardtype;        sum + = Data.sum;        Photonum + = Data.photonum;        Reblognum + = Data.reblognum;        Postnum + = Data.postnum;        Videonum + = Data.videonum; MusicnuM + = Data.musicnum;        Videoshortnum + = Data.videoshortnum;        Questionnum + = Data.questionnum;        Appnum + = Data.appnum;        Dialognum + = Data.dialognum;                if (!reblogflag) {if (Dashboardflag) {sum + = 1;                if (Dashboardtype = =) {Postnum + = 1;                } else if (Dashboardtype = =) {Photonum + = 1;                } else if (Dashboardtype = =) {Videonum + = 1;                } else if (Dashboardtype = =) {Videoshortnum + = 1;                } else if (Dashboardtype = =) {Musicnum + = 1;                } else if (Dashboardtype = =) {Questionnum + = 1;                } else if (Dashboardtype = =) {Appnum + = 1;                } else if (Dashboardtype = =) {Dialognum + = 1;                }} else {if (Dashboardtype = = 20) {    Photonum + = 1;        }}} and else if (Reblogflag && dashboardflag) {reblognum + = 1; }} return {Sum:numberint (sum), Reblognum:numberint (Reblognum), Postnum:numberint (Postnum), Photonum:numberint ( Photonum), Videonum:numberint (Videonum), Videoshortnum:numberint (Videoshortnum), Musicnum:numberint (MusicNum) , Questionnum:numberint (Questionnum), Appnum:numberint (Appnum), Dialognum:numberint (Dialognum)}; Db.getmongo (). Setslaveok ();d b.dashboard_basic.mapreduce (map, reduce, {out:{merge: ' Aaccou Nttemp '}}); var results = Db.aAccounttemp.find ();//Reorganize data format into normal table while (Results.hasnext ()) {var obj = ResU    Lts.next ();    var value = Obj.value;    var sum = numberint (value.sum);    var reblognum = Numberint (value.reblognum);    var postnum = Numberint (value.postnum);    var photonum = Numberint (value.photonum);    var videonum = Numberint (value.videonum); var videoshortnum = NUmberint (Value.videoshortnum);    var musicnum = Numberint (value.musicnum);    var questionnum = Numberint (value.questionnum);    var appnum = Numberint (value.appnum);    var dialognum = Numberint (value.dialognum);    var accountId = obj._id; Db.dashboard_account_num.insert ({accountid:accountid,sum:sum,reblognum:reblognum,postnum:postnum,photonum: Photonum, Videoshortnum:videoshortnum,videonum:videonum,musicnum:musicnum,questionnum:questionnum, AppNum:ap Pnum,dialognum:dialognum});}   Print (' success Insert total ' + results.count () + ' datas ');d B.aaccounttemp.drop () quit ()