At the beginning of September, rmongodb officially released a revised version, which means that the language used for numerical computing can also be integrated with nosql products, however, since no company around me is actually using the combination of R and MongoDB, we did not dare to take it lightly in terms of efficiency, so we did such a test.
The test environment is 8-core and 64-bit. The library used for testing is a collection of about 30 GB without sharding. Stores user preferences, tags, and other data.
1 Library (rmongodb)
2
3 Mongo <-Mongo. Create ()
4
5 If (Mongo. Is . Connected (Mongo ))
6 {
7 NS <- ' Rivendell. User '
8 Print ( ' Query a field without an index. ' )
9 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list (Friend = 600 ))))
10 Print ( ' Query one unindexed field, multiple, without Buffer ' )
11 Print (System. Time (P <-Mongo. Find (Mongo, NS, list (Friend = 600 ))))
12 Print ( ' Check whether a cache policy exists. ' )
13 Print (System. Time (P <-Mongo. Find (Mongo, NS, list (Friend = 600 ))))
14
15 Print ( ' Query one unindexed field, multiple, has Buffer ' )
16 Buf <-Mongo. bson. Buffer. Create ()
17 Mongo. bson. Buffer. append (BUF, ' Friend ' , 600l)
18 Query <-Mongo. bson. From . Buffer (BUF)
19 Print (System. Time (P <-Mongo. Find (Mongo, NS, query )))
20 Print ( ' Check whether a cache policy exists. ' )
21 Buf <-Mongo. bson. Buffer. Create ()
22 Mongo. bson. Buffer. append (BUF, ' Friend ' , 600l)
23 Query <-Mongo. bson. From . Buffer (BUF)
24 Print (System. Time (P <-Mongo. Find (Mongo, NS, query )))
25
26 Print ( ' Query a record greater ' )
27 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list (Friend = List ( ' $ GT ' = 600l )))))
28 Print ( ' More than one record, query multiple records ' )
29 Print (System. Time (cursor <-Mongo. Find (Mongo, NS, list (Friend = List ( ' $ GT ' = 600l )))))
30 Mongo. cursor. Destroy (cursor)
31
32 Print ( ' Query an indexed record ' )
33 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list ( ' _ Id ' = 3831809l ))))
34 Print ( ' Query index records ' )
35 Print (System. Time (P <-Mongo. Find (Mongo, NS, list ( ' _ Id ' = 3831809l ))))
36
37 Print ( ' Insert a record ' )
38 Buf <-Mongo. bson. Buffer. Create ()
39 Mongo. bson. Buffer. append (BUF, ' Name ' , " Huangxin " )
40 Mongo. bson. Buffer. append (BUF, ' Age ' , 22L)
41 P <-Mongo. bson. From . Buffer (BUF)
42 Print (System. Time (Mongo. insert (Mongo, NS, p )))
43
44 Print ( ' Locate the inserted record ' )
45 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list ( ' Name ' = ' Huangxin ' ))))
46 If (! Is . Null (p ))
47 {
48 Print ( ' Success ' )
49 }
50
51 Print ( ' Batch insert ' )
52
53 Buf <-Mongo. bson. Buffer. Create ()
54 Mongo. bson. Buffer. append (BUF, ' Name ' , ' Huangxin ' )
55 Mongo. bson. Buffer. append (BUF, ' Age ' , 22L)
56 P1 <-Mongo. bson. From . Buffer (BUF)
57
58 Buf <-Mongo. bson. Buffer. Create ()
59 Mongo. bson. Buffer. append (BUF, ' Name ' , ' Huangxin ' )
60 Mongo. bson. Buffer. append (BUF, ' Age ' , 22L)
61 P2 <-Mongo. bson. From . Buffer (BUF)
62
63 Buf <-Mongo. bson. Buffer. Create ()
64 Mongo. bson. Buffer. append (BUF, ' Name ' , ' Huangxin ' )
65 Mongo. bson. Buffer. append (BUF, ' Age ' , 22L)
66 P3 <-Mongo. bson. From . Buffer (BUF)
67
68 Print (System. Time (Mongo. Insert. Batch (Mongo, NS, list (P1, P2, P3 ))))
69
70 Print ( ' Locate the records just inserted in batches ' )
71 Print (System. Time (cursor <-Mongo. Find (Mongo, NS, list ( ' Name ' = ' Huangxin ' ))))
72
73 I <-0
74 While (Mongo. cursor. Next (cursor ))
75 {
76 I <-I + 1
77 }
78 Print (I)
79
80 Print ( ' Batch update ' )
81 Print (System. Time (Mongo. Update (Mongo, NS, list (name = ' Huangxin ' ), List ( ' Name ' = ' Kym ' ))))
82
83 Print ( ' Check whether the update is successful ' )
84 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list ( ' Name ' = ' Kym ' ))))
85 If (! Is . Null (p ))
86 {
87 Print ( ' Success ' )
88 }
89
90 Print ( ' Batch Delete ' )
91 Print (System. Time (Mongo. Remove (Mongo, NS, list (name = ' Kym ' ))))
92 }
93
94 Print (System. Time (P <-Mongo. Find. One (Mongo, NS, list ( ' Name ' = ' Kym ' ))))
95 If (! Is . Null (p ))
96 {
97 Print ( ' Success ' )
98 }
[1] " Query a field without an index. "
User System elapsed
0.000 0.000 0.115
[1] " Query one unindexed field, multiple, without Buffer "
User System elapsed
0.000 0.000 32.513
[1] " Check whether a cache policy exists. "
User System elapsed
0.000 0.000 32.528
[1] " Query one unindexed field, multiple, has Buffer "
User System elapsed
0.000 0.000 32.685
[1] " Check whether a cache policy exists. "
User System elapsed
0.000 0.000 33.172
[1] " Query a record greater "
User System elapsed
0.000 0.000 0.001
[1] " More than one record, query multiple records "
User System elapsed
0.000 0.000 0.014
[1] " Query an indexed record "
User System elapsed
0 0 0
[1] " Query index records "
User System elapsed
0 0 0
[1] " Insert a record "
User System elapsed
0 0 0
[1] " Locate the inserted record "
User System elapsed
0.00 0.00 35.42
[1] " Success "
[1] " Batch insert "
User System elapsed
0 0 0
[1] " Locate the records just inserted in batches "
User System elapsed
0.004 0.000 35.934
[1] 7
[1] " Batch update "
User System elapsed
0.000 0.004 0.000
[1] " Check whether the update is successful "
User System elapsed
0.000 0.000 67.773
[1] " Success "
[1] " Batch Delete "
User System elapsed
0 0 0
User System elapsed
0.000 0.000 91.396
What I have never understood before is why the difference between the two is greater than or equal to the difference. Later, when I used python for the same test, I found that the efficiency of python is actually the same, so this proves that this is not a MongoDB problem, I don't believe that at the database level, the driver of a language is very different.
Later I found a difference between Python and R about MongoDB driver. First, in Python find, instead of pulling back the entire queried dataset, a cursor is returned, that is, it does not consume time when executing the find command. when next (), it will actually execute this query.
But R is different. R will first consider the size of the dataset (or other situations), and then return the cursor as needed or pull the entire dataset back. If we convert the previous while Mongo. cursor. next (cursor) is also calculated in time, so we will find that, in fact, the difference in efficiency is not obvious for operations greater than or equal .......
In practice, batch insert is a very common application scenario, but for R or Matlab language, the efficiency of the cycle is always hard, so next, I will try to use the apply series to solve the loop problem of the r language. If the actual operation is found feasible, next we will try to use parallel computing libraries such as mutilab to make full use of the multi-core efficiency!
