記一次MongoDB Map&Reduce入門操作

標籤：

• 需求說明

用Map&Reduce計算幾個班級中，每個班級10歲和20歲之間學生的數量：

• 需求分析
• 學生表的欄位：

db.students.insert({classid:1, age:14, name:‘Tom‘})

將classid隨機1和2、age在8-25歲之間隨機，name在3-7個字元之間隨機。

• 資料寫入
• 資料寫入java指令碼

往mrtask庫中students寫入1000萬條資料：

 package org.test; import java.util.ArrayList;import java.util.List;import java.util.Random;import com.mongodb.BasicDBObject;import com.mongodb.DB;import com.mongodb.DBCollection;import com.mongodb.DBCursor;import com.mongodb.DBObject;import com.mongodb.MongoClient;import com.mongodb.ServerAddress; public class TestMongoDBReplSet {     public static void main(String[] args) {        try {            List<ServerAddress> addresses = new ArrayList<ServerAddress>();            ServerAddress address1 = new ServerAddress("172.16.16.89", 27017);            addresses.add(address1);            MongoClient client = new MongoClient(addresses);            DB db = client.getDB("mrtask");            DBCollection coll = db.getCollection("students");             // 資料寫入            BasicDBObject object = new BasicDBObject();            for (int i = 1; i <= 10000000; i++) {                object.append("classid", 1 + (int) (Math.random() * 2));                object.append("age", 8 + (int) (Math.random() * 17));                object.append("name", getName());                coll.insert(object);                object.clear();            }        } catch (Exception e) {            e.printStackTrace();        }     }     public static String getName() {        ArrayList list = new ArrayList();        for (char c = ‘a‘; c <= ‘z‘; c++) {            list.add(c);        }        String str = "";        int end = 3 + (int) (Math.random() * 4);        for (int i = 0; i < end; i++) {            int num = (int) (Math.random() * 26);            str = str + list.get(num);        }        return str;    } }

　　

• 查看資料寫入

經查看，mrtask庫中students表中有1000萬條的資料：

[[email protected] bin]# ./mongo

MongoDB shell version: 2.6.11

connecting to: test

> show dbs

admin   (empty)

local   0.078GB

mrtask  3.952GB

test    0.453GB

> use mrtask

switched to db mrtask

> db.students.find().count()

10000000

 

• Map&Reduce計算
• Map計算

> mapfun = function(){emit(this.classid,1)}

• Reduce計算

> reducefun=function (key, values) { var count = 0; values.forEach(function (v) {count += v;}); return count; }

> ff = function (key, value) { return {classid:key, count:value}; }

• Result輸出

> classid_res = db.runCommand({

mapreduce:"students",

map:mapfun,

reduce:reducefun,

out:"students_classid_res",

finalize:ff,

query:{age:{$gt:10,$lt:20}}

});

 

• 計算結果

> db.students_classid_res.find()

{ "_id" : 1, "value" : { "classid" : 1, "count" : 2643128 } }

{ "_id" : 2, "value" : { "classid" : 2, "count" : 2650870 } }

 

記一次MongoDB Map&Reduce入門操作